Content deleted Content added
→Servers that support HTTP compression: irrelevant, the older module is deprecated anyway |
|||
(23 intermediate revisions by 17 users not shown) | |||
Line 1:
{{Short description|Capability that can be built into web servers and web clients
{{HTTP}}
'''HTTP compression''' is a capability that can be built into [[web server]]s and [[web client]]s to improve transfer speed and bandwidth utilization.<ref>{{cite web|url=http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/d52ff289-94d3-4085-bc4e-24eb4f312e0e.mspx?mfr=true|title=Using HTTP Compression (IIS 6.0)|access-date=9 February 2010|publisher=Microsoft Corporation}}</ref>
HTTP data is [[Data compression|compressed]] before it is sent from the server: compliant browsers will announce what methods are supported to the server before downloading the correct format; browsers that do not support compliant compression method will download uncompressed data. The most common compression schemes include [[gzip]] and [[Brotli]];
There are two different ways compression can be done in HTTP. At a lower level, a Transfer-Encoding header field may indicate the payload of an HTTP message is compressed. At a higher level, a Content-Encoding header field may indicate that a resource being transferred, [[Web cache|cached]], or otherwise referenced is compressed. Compression using Content-Encoding is more widely supported than Transfer-Encoding, and some browsers do not advertise support for Transfer-Encoding compression to avoid triggering bugs in servers.<ref>[https://code.google.com/p/chromium/issues/detail?id=94730 'RFC2616 "Transfer-Encoding: gzip, chunked" not handled properly'], [[Chromium (browser)|Chromium]] Issue 94730</ref>
==Compression scheme negotiation==
The negotiation is done in two steps, described in RFC 2616 and RFC 9110:
1. The [[web client]] advertises which compression schemes it supports by including a list of tokens in the [[HTTP request]]. For ''Content-Encoding'', the list is in a field called ''Accept-Encoding''; for ''Transfer-Encoding'', the field is called ''TE''.
Line 35:
The official list of tokens available to servers and client is maintained by IANA,<ref>{{cite web|url=https://www.iana.org/assignments/http-parameters/http-parameters.xhtml#content-coding|title=Hypertext Transfer Protocol Parameters - HTTP Content Coding Registry|publisher=IANA|access-date=18 April 2014}}</ref> and it includes:
*br – [[Brotli]], a compression algorithm specifically designed for HTTP content encoding, defined in
*[[compress]] – UNIX "compress" program method (historic; deprecated in most applications and replaced by gzip or deflate)
*deflate – compression based on the [[DEFLATE|deflate]] algorithm (described in
*exi – W3C [[Efficient XML Interchange]]
*[[gzip]] – GNU zip format (described in
*[[Identity function|identity]] – No transformation is used. This is the default value for content coding.
*[[Pack200|pack200-gzip]] – Network Transfer Format for Java Archives<ref>{{cite web|url=https://jcp.org/en/jsr/detail?id=200|title=JSR 200: Network Transfer Format for Java Archives|publisher=The Java Community Process Program}}</ref>
*[[zstd]] –
In addition to these, a number of unofficial or non-standardized tokens are used in the wild by either servers or clients:
Line 49:
*[[Lempel–Ziv–Markov_chain_algorithm|lzma]] – compression based on (raw) LZMA is available in Opera 20, and in elinks via a compile-time option<ref>[http://elinks.or.cz/documentation/html/manual.html-chunked/ch01s07.html#CONFIG-LZMA elinks LZMA decompression]</ref>
*peerdist<ref>{{cite web|url=http://msdn.microsoft.com/en-us/library/dd304322%28v=PROT.10%29.aspx|title=[MS-PCCRTP]: Peer Content Caching and Retrieval: Hypertext Transfer Protocol (HTTP) Extensions|publisher=Microsoft|access-date=19 April 2014}}</ref> – Microsoft Peer Content Caching and Retrieval
*[[rsync]]<ref>{{cite web |title=rproxy: Protocol Definition for HTTP rsync Encoding |url=https://rproxy.samba.org/doc/protocol/protocol.html |website=rproxy.samba.org}}</ref>
*xpress
*[[XZ Utils|xz]]
==Servers that support HTTP compression==
Line 73:
*[[Varnish (software)|Varnish]] – built-in. Works also with [[Edge Side Includes|ESI]]
*[https://line.github.io/armeria/ Armeria] – Serving pre-compressed files<ref>{{cite web|url=https://line.github.io/armeria/server-http-file.html#serving-pre-compressed-files|title=Serving static files part of Armeria's documentation}}</ref>
*[[NaviServer]]
*[[Caddy (web server)|Caddy]] – built-in via [https://caddyserver.com/docs/caddyfile/directives/encode encode]
Many [[content delivery network]]s also implement HTTP compression to improve speedy delivery of resources to end users.
The compression in HTTP can also be achieved by using the functionality of [[server-side scripting]] languages like [[PHP]], or programming languages like [[Java (programming language)|Java]].
Various online tools exist to verify a working implementation of HTTP compression. These online tools usually request multiple variants of a URL, each with different request headers (with varying Accept-Encoding content). HTTP compression is considered to be implemented correctly when the server returns a document in a compressed format.<ref>{{ cite web|url=https://httptools.dev/gzip-brotli-check|title=How does the gzip compression check work? }} httptools.dev, retrieved 10 April 2022.</ref> By comparing the sizes of the returned documents, the effective compression ratio can be calculated (even between different compression algorithms).
==Problems preventing the use of HTTP compression==
A 2009 article by Google engineers Arvind Jain and Jason Glasgow states that more than 99 person-years are wasted<ref name="google-use-compression">{{cite web|url=https://developers.google.com/speed/articles/use-compression|title=Use compression to make the web faster|access-date=22 May 2013|publisher=Google
Another problem found while deploying HTTP compression on large scale is due to the '''deflate''' encoding definition: while HTTP 1.1 defines the '''deflate''' encoding as data compressed with deflate (RFC 1951) inside a [[zlib]] formatted stream (RFC 1950), Microsoft server and client products historically implemented it as a "raw" deflated stream,<ref>{{cite web|url=https://stackoverflow.com/questions/9170338/why-are-major-web-sites-using-gzip/9186091#9186091|title=deflate - Why are major web sites using gzip?|publisher=Stack Overflow|access-date=18 April 2014}}</ref> making its deployment unreliable.<ref>{{cite web|url=http://www.vervestudios.co/projects/compression-tests/|title=Compression Tests: About|publisher=Verve Studios|archive-url=https://web.archive.org/web/20150102111552/http://www.vervestudios.co/projects/compression-tests/|archive-date=2 January 2015|access-date=18 April 2014}}</ref><ref>{{cite web|url=http://zoompf.com/blog/2012/02/lose-the-wait-http-compression|title=Lose the wait: HTTP Compression|publisher=Zoompf Web Performance|access-date=18 April 2014}}</ref> For this reason, some software, including the Apache HTTP Server, only implement '''gzip''' encoding.
Line 98 ⟶ 102:
==External links==
*
*{{IETF RFC|9110|link=no}}: HTTP Semantics
*[https://www.iana.org/assignments/http-parameters HTTP Content-Coding Values] by Internet Assigned Numbers Authority
*[http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs:Modcompress Compression with lighttpd]
*[http://www.codinghorror.com/blog/2004/08/http-compression-and-iis-6-0.html Coding Horror: HTTP Compression on IIS 6.0] {{Webarchive|url=https://web.archive.org/web/20140206020708/http://www.codinghorror.com/blog/2004/08/http-compression-and-iis-6-0.html |date=2014-02-06 }}
*{{webarchive |url=https://web.archive.org/web/20110716033901/http://www.15seconds.com/Issue/020314.htm |date=July 16, 2011 |title=15 Seconds: Web Site Compression }}
*[http://www.serverwatch.com/tutorials/article.php/3514866 Using HTTP Compression] {{Webarchive|url=https://web.archive.org/web/20160314155152/http://www.serverwatch.com/tutorials/article.php/3514866 |date=2016-03-14 }} by Martin Brown of Server Watch▼
▲*[http://www.serverwatch.com/tutorials/article.php/3514866 Using HTTP Compression] by Martin Brown of Server Watch
*[https://web.archive.org/web/20060411174003/http://www.devshed.com/c/a/PHP/Using-HTTP-Compression-in-PHP-Make-Your-Web-Pages-Load-Faster/ Using HTTP Compression in PHP]
*[https://web.archive.org/web/20120430023716/https://banu.com/blog/38/dynamic-and-static-http-compression-with-apache-httpd/ Dynamic and static HTTP compression with Apache httpd]
|