From d8518b70b912aa55fc47400173bf6229e40b71d0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C5=A0imon=20Luka=C5=A1=C3=ADk?= <isimluk@fedoraproject.org>
Date: Wed, 8 Jul 2020 15:17:31 +0200
Subject: [PATCH] Make a use of HTTP header content-encoding: gzip if available

When fetching remote resources, some servers/CDNs may be able to serve us
compressed http response even in cases when the original file is not compressed
XML. libcurl is able to process encoded html for us with no added maintenance
costs.

Attached please find a CURL log of fetching plain XML file from Red Hat CDN:

Downloading: https://www.redhat.com/security/data/oval/com.redhat.rhsa-RHEL7.xml
...
*   Trying 104.90.105.254:443...
* Connected to www.redhat.com (104.90.105.254) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: businessCategory=Private Organization; jurisdictionC=US; jurisdictionST=Delaware; serialNumber=2945436; C=US; ST=North Carolina; L=Raleigh; O=Red Hat, Inc.; CN=www.redhat.com
*  start date: Feb 24 00:00:00 2020 GMT
*  expire date: May 24 12:00:00 2022 GMT
*  subjectAltName: host "www.redhat.com" matched cert's "www.redhat.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 Extended Validation Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x776c3b0)
> GET /security/data/oval/com.redhat.rhsa-RHEL7.xml HTTP/2
Host: www.redhat.com
accept: */*
accept-encoding: gzip

* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< server: Apache
< last-modified: Wed, 08 Jul 2020 12:41:28 GMT
< etag: "7f694279-fca5e0-5a9ed6d376a08"
< accept-ranges: bytes
< content-type: text/xml
< content-encoding: gzip
< content-length: 1766376
< date: Wed, 08 Jul 2020 13:15:29 GMT
< vary: Accept-Encoding
< strict-transport-security: max-age=31536000
<
* Connection #0 to host www.redhat.com left intact
---
 src/common/oscap_acquire.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/common/oscap_acquire.c b/src/common/oscap_acquire.c
index 60ab62c05..551da43f0 100644
--- a/src/common/oscap_acquire.c
+++ b/src/common/oscap_acquire.c
@@ -302,6 +302,7 @@ char* oscap_acquire_url_download(const char *url, size_t* memory_size)
 	curl_easy_setopt(curl, CURLOPT_URL, url);
 	curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_to_memory_callback);
 	curl_easy_setopt(curl, CURLOPT_WRITEDATA, buffer);
+	curl_easy_setopt(curl, CURLOPT_ACCEPT_ENCODING, "");
 	curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, true);
 
 	CURLcode res = curl_easy_perform(curl);