[squid-users] TCP_REFRESH_UNMODIFIEDs are really MISSes

[Alex Rousskov]

On 10/11/2017 04:52 AM, Amos Jeffries wrote:
> On 11/10/17 23:31, Victor Sudakov wrote:
>> Alex Rousskov wrote:
>>> To be sure that you are comparing apples to apples, you would need
>>> to log the X-Cache-Lookup response header to access.log (or enable
>>> ALL,2 debugging to see headers and/or collect packet traces).

>> I don't think the freebsd-update cares about the X-Cache-Lookup
>> response, I'm just trying to save bandwidth and don't quite like the
>> result.

> The X-Cache* headers are debug output from Squid. [...]
> MISS means a server was contacted for _something_ and HIT means local
> cache only was used.


Here is a more accurate description of the X-Cache and X-Cache-Lookup
header fields:

X-Cache-Lookup:MISS means that Squid did not find the requested object
in its cache after parsing the client request, and X-Cache-Lookup:HIT
means that it did. In either case, the X-Cache-Lookup header field does
not tell us whether the server was contacted afterwards, although:

* X-Cache-Lookup:MISS implies that the server contact is very likely
(but various errors and other corner cases may prevent any server
contact) and

* X-Cache-Lookup:HIT implies that the server contact is less likely (but
client revalidation requests, stale cached responses, and other cases
make such contact possible).

* X-Cache-Lookup:NONE means that there was no cache lookup at all.
(Listed here for completeness sake)


There is also X-Cache header field, but its value is just a mapping of
the final(?) status code to HIT or MISS categories. In Squid v3.5, the
HIT category is assigned to TCP_HIT, TCP_IMS_HIT, TCP_INM_HIT,
TCP_REFRESH_FAIL_OLD, TCP_REFRESH_UNMODIFIED, TCP_NEGATIVE_HIT,
TCP_MEM_HIT, and TCP_OFFLINE_HIT statuses. The rest are categorized as
MISSes. The X-Cache header field does not tell us whether the server was
contacted.



To avoid misunderstanding, I have provided this information to help you
triage why Squid revalidates so much in your use case, and not to
justify Squid behavior or blame the freebsd-update client. You now know
what the logged status and the header values mean. And you now know that
you have to look at the same transaction to analyze the combination of
logged values and headers. If you want to know _why_ Squid does what it
does, the information Amos and I have provided may help you dig deeper.
Once you know "why", you may be able to adjust Squid configuration (or
something else!) to reduce excessive revalidations.


HTH,

Alex.
P.S. If the above is not documented on Squid wiki, please consider
documenting it. Thank you in advance.