[squid-users] compression in Squid

[Amos Jeffries]

On 24/11/2016 2:50 a.m., Ralf Hildebrandt wrote:
> * Amos Jeffries wrote:
>> So I have finally got around to implementing the proxy compression
>> feature in HTTP.
>>
>> The code and more info is available at:
>> <https://code.launchpad.net/~yadi/squid/te-gzip>
>>
>> This Squid built with zlib can right now receive and decompress on the
>> fly traffic from a server which uses Transfer-Encoding: deflate.
> 
> I don't understand this feature. Why is the compressed data not simply
> passed on to the client?
> 

When fully implemented that is one of the things that should be
possible. One small-ish step at a time ;-P

If it helps:

The difference between Content-Encoding and Transfer-Encoding is a fine
line. Basically coming down to a matter of whether data is encoded at
rest or in transit, and what types of agent are able to safely
encode/decode.

* Transfer-Encoding is for encoding data _only_ while in transit.

The recipient decodes/decompresses it before making use of it. If the
client advertises T-E support for the used algorithms and the data would
just be relayed anyway, it does not have to be decoded. But I have not
implemented that at this stage.

* Content-Encoding is for data at rest _and_ in transit.

C-E assumes the data is compressed out-of-band, gets stored that way on
the origin and stays that way end-to-end until it reaches the client
which sent the Accept-Encoding for that algorithm. That also implies
that metadata about the compression can be delivered out-of-band so it
is somewhat risky for proxies to play with.


>> The primary use-case is for Squid installations where two proxies are
>> used to reduce bandwidth over a slow or expensive link (ie satellite).
> 
> So the proxies are compressing everything (between them? between proxy
> and internet? between client and proxy?)
> 

Between them for now. If possible across as many hops as support the
feature.

More options for both encoding algorithms and ways to pass the data
around are on the todo list once the basics are working.

Amos