[squid-users] TCP_REFRESH_UNMODIFIEDs are really MISSes

[Victor Sudakov]

Amos Jeffries wrote:
> > Alex Rousskov wrote:
> >> On 10/10/2017 07:50 PM, Victor Sudakov wrote:
> >>
> >>> When updating several almost identical FreeBSD hosts via a
> >>> squid-3.5.27 proxy, I expect to see lots of HITs, because the patches
> >>> on the update server should be identical too.
> >>>
> >>> However, I see lots of TCP_REFRESH_UNMODIFIED lines, and only
> >>> occasional HIT statuses.
> >>
> >>
> >> There are many kinds of "hits":
> >>
> >> * If your primary concern is data transfer/bandwidth usage, then you
> >> should focus on so called byte hits. TCP_REFRESH_UNMODIFIED is
> >> essentially one of those because the origin server sends a tiny
> >> headers-only response to Squid.
> > 
> > My primary concern is bandwidth usage. You are saying that the
> > TCP_REFRESH_UNMODIFIED transaction means that the bulk of the data is
> > not transferred from Webserver to Squid, only the headers are fetched?
> 
> Yes.
> 
> > 
> > If you look at this line:
> > 
> > 1507630625.515    448 212.73.124.12 TCP_REFRESH_UNMODIFIED/200 980593 GET http://update6.freebsd.org/to-10.4-RELEASE/i386/bp/1195d725efce75f0db6317f507d8a9514f35613c7466673a86db23100bd6fa77-c4150431df5b14b15546b3db5e30f2bd2c35d6d52812f057dc101fab20847324 - HIER_DIRECT/198.148.79.66 application/octet-stream
> > 
> > There cannot be 980593 bytes of headers? What kind of gigantic header
> > does it make?
> > 
> 
> That is header only between Squid and server (REFRESH_UNMODIFIED), with 
> full object delivered to the client (final status "200").

I see now, the 980593 bytes in the log is between Squid and client, not between Squid
and Webserver.

> >>
> >> According to your pasted log and HTTP messages, the GET request URL that
> >> resulted an "X-Cache-Lookup: MISS" response header does not match any of
> >> the logged URLs; even the host part of the URL does not match.
> > 
> > Of course, there are just unrelated examples. There are thousands of
> > fetches from update*.freebsd.org
> 
> So you just posted them to waste everybody on this lists bandwidth? 

Hmm, well, no. Consider them two separate examples: a) a sample log
output and b) a sample HTTP session. Both were necessary for
illustrating the question. I never thought someone would try to
juxtapose them.

> Thanks, and kind of ironic given your "problem".

About 30 lines of logs and Wireshark output surely wasted the lists
bandwidth. Sorry about that.

> 
> > 
> >> To be sure that you are comparing apples to apples, you would need
> >> to log the X-Cache-Lookup response header to access.log (or enable
> >> ALL,2 debugging to see headers and/or collect packet traces).
> > 
> > I don't think the freebsd-update cares about the X-Cache-Lookup
> > response, I'm just trying to save bandwidth and don't quite like the
> > result.
> > 
> 
> The X-Cache* headers are debug output from Squid. Of course the updater 
> software does not care about it ... you do, sort of anyway since it 
> shows whether server bandwidth was involved. They do not show REFRESH, 
> so MISS means a server was contacted for _something_ and HIT means local 
> cache only was used.

Thank you for clarification. I will not fear the
TCP_REFRESH_UNMODIFIEDs any more.

-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
AS43859