[squid-users] cache hit rate isn't what I'd expect

Thu Sep 28 23:54:09 UTC 2017

Here ya go

26/Sep/2017:20:10:27    137 10.93.3.47 TCP_HIT/200 11265 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr - HIER_NONE/-
26/Sep/2017:20:10:33     46 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:10:42      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:10:47      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:10:52      5 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:10:56    234 10.93.3.47 TCP_HIT/200 11265 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr - HIER_NONE/-
26/Sep/2017:20:11:11      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:15      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:19      6 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:24      5 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:28      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:32      1 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:37      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:41      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:48      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:53      4 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:11:57      6 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:01      7 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:06      5 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:10      4 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:14     11 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:19      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:23      6 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:28      4 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:32      6 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:37     96 10.93.3.47 TCP_HIT/200 11265 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr - HIER_NONE/-
26/Sep/2017:20:12:41      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:12:49    225 10.93.3.47 TCP_HIT/200 11266 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr - HIER_NONE/-
26/Sep/2017:20:12:59      0 10.93.3.47 TCP_HIT/200 11265 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr - HIER_NONE/-
26/Sep/2017:20:13:03      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:13:08      0 10.93.3.47 TCP_HIT/200 11265 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr - HIER_NONE/-
26/Sep/2017:20:13:13      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:13:27      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:13:33      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:13:37      4 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:13:42    185 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:13:47    349 10.93.3.47 TCP_HIT/200 11265 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr - HIER_NONE/-
26/Sep/2017:20:13:52     21 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:13:56      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:14:00      7 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:14:06     72 10.93.3.47 TCP_HIT/200 11266 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr - HIER_NONE/-
26/Sep/2017:20:14:36      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:14:41      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:14:45      2 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
26/Sep/2017:20:14:54      3 10.93.3.47 TCP_MISS/200 11259 GET
https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr -
HIER_DIRECT/192.229.163.180
--
Aaron Turner
https://synfin.net/         Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality.  "Something cannot emerge from nothing,"
he said.  This is profound thinking if you understand how unstable
"the truth" can be.  -- Frank Herbert, Dune


On Thu, Sep 28, 2017 at 3:47 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:
> On 29/09/17 11:29, Aaron Turner wrote:
>>
>> So this grep through my access logs for this single URL does a good
>> job illustrating a rather interesting problem:
>>
>> $ grep -h 'https://static.licdn.com/sc/h/ddzuq7qeny6qn0ysh3hj6pzmr
>> text/css ip_index=0,client=m0078269' access.*.log | sort
>>
>>
> ...
>>
>>
>> At first I thought this was because the because I have a bunch of
>> clients, each of which behaves exactly the same except for one thing:
>> the client includes a unique request header that squid strips off
>> before forwarding to the server (you can see it logged as
>> client=mXXXXX_XXXX).  But in this case I've controlled for that and
>> only grep'd for a single client's request.  I've even tried setting
>> "vary_ignore_expire on", but that doesn't seem to be a complete fix.
>>
>> I can't for the life of me understand why the low hit rate though.
>>
>
> The duration and size fields are quite useful for detecting reasons for
> HIT/MISS.
>
> Request headers should not affect the response caching, unless they are
> listed in the servers Vary header.
>
> In this case the server is delivering broken Vary responses. redbot.org says
> it is using Vary:Accept-Encoding sometimes, so both the Vary and
> Accept-Encoding would be useful info to log.
>
> I expect it is the usual problem of clients fighting over whose variant gets
> cached when this type of server breakage happens - when the Vary header
> changes or disappears, old variants become unfindable until it changes back.
>
> Amos
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users