[squid-users] Vary object loop returns

[Amos Jeffries]

On 8/06/2016 8:21 a.m., Heiler Bemerguy wrote:
> 
> Some servers will reply like this, trying to avoid caching at any cost
> (I think):
> 
> HTTP/1.1 200 OK
> Server: nginx
> Content-Type: image/x-icon
> Last-Modified: Tue, 07 Jun 2016 11:16:55 GMT
> ETag: "5756ad27-47e"
> Content-Length: 1150
> X-Suppressed-Cache-Control: max-age=600
> Cache-Control: private, max-age=0, must-revalidate
> X-Suppressed-Expires: Tue, 07 Jun 2016 20:07:36 GMT
> Expires: Thu, 01 Jan 1970 00:00:00 GMT
> Date: Tue, 07 Jun 2016 19:57:36 GMT
> X-Varnish: 510207311
> *Vary:
> Accept,If-None-Match,If-Modified-Since,Accept-Language,Accept-Encoding,X-Client-Locale,User-Agent,X-Device*
> 
> 
> Then our squid will create a vary object with all that information,
> giving this bomb: httpMakeVaryMark:
> accept="image%2Fpng,image%2F*%3Bq%3D0.8,*%2F*%3Bq%3D0.5",
> if-none-match="%225756ad27-47e%22", if-modified-since,
> accept-language="en-US,en%3Bq%3D0.8,pt-BR%3Bq%3D0.5,pt%3Bq%3D0.3",
> accept-encoding="none", x-client-locale,
> user-agent="Mozilla%2F5.0%20(Windows%20NT%2010.0%3B%20WOW64%3B%20rv%3A46.0)%20Gecko%2F20100101%20Firefox%2F46.0",
> x-device
> 
> It's squid "fault" to convert spaces and symbols to %values, and I think
> no sanity check is performed on it..

There are at least three sanity checks/protections performed.

 1) the client request headers must parse according to HTTP rules. If
any one of them dont Squid either wont get this far, or will be treating
that header as non-existent/empty.

 2) as of 3.5.19 the total length of the expansion must be short enough
to store in a limited string buffer (~64KB). The recent CVE-2016-* set
of issues happened because it was possible to get past this sometimes.

 3) the header field values are RFC1738 encoded as they are added to the
sub-key to ensure it complies with HTTP octet requirements and prevent
log based attacks. Thats the %-encoding you see in the logs.


> still, I don't see the code where
> it checks if all this info from the new client is identical to the
> stored one.. and I don't know where the "loop" comes from...

That is a set of separate checks deeper inside store itself which are
performed for every lookup, not just Vary sub-key lookups. One of which
is to compare the URL of the found object with the active request. The
second of which you found with your extra debug already.

Amos