[squid-users] Need better debug_options values to track down cache MISSes

[Greg]

Hi all,

so, there's my proxy problem I couldn't crack, even after spending 2+
days tweaking-googling-debugging. :(

The problem: my _new_ Squid installation (Ubuntu 14 LTS with Squid 3.3.8)
won't cache most pages the old Squid does (old Fedora with Squid 3.1.15).

Infos:
- Simple reverse proxy, port 80 only, one uplink straight to the app
servers. (The old one points to a load balancer. For testing purposes and
seeing the mail on "Thu, 4 Apr 2013 16:54:10 -0700", I'm pointing the new
one to a specific app server.)
- Pretty standard configuration, standard refresh_patterns. (I'd be
surprised if it was a config problem since ICOs are cached well.)
- I have disk caching on the new one, the disk cache has been initialised
with -z.
- The old one has a very relaxed iptables, the new one has ufw, later set
to allow port 80 and 22 only for incoming connections.
- Clocks are in sync using ntpd - the old one has +20min delay (ie. shows a
later time than now). Since it's the live one, it's working and it handles
a lot of load well, I haven't dared to correct it.
- The new Squid gives HITs for favicon.ico and RSS feeds only - these have
different headers and mimetypes compared to HTML files (and that's by
design).
- This new server seems to save accessed HTML files in the disk cache but
still gives X-Cache-Lookup:MISS for all of them later.
- I debugged using debug_options and logging: headers look okay
(Cache-Control, Pragma, Vary - pasted below), stale/fresh calculation looks
okay (pages are deemed fresh).
- The best I got so far is this: "client_side_reply.cc(1618)
identifyFoundObject: clientProcessRequest2: StoreEntry is NULL -  MISS",
even for pages I see in the disk cache.
- I don't want to use overriding in refresh_patterns, because I don't
understand the problem yet, I don't know what problem would this hack hide
(and for how long).
- I don't want to move away from the current platform and version if
possible because this has LTS support and we'll have many proxy servers if
we finally figure out the right config.
- Strange thing, the old proxy seems to return stuff with HTTP/1.0 and the
new one with HTTP/1.1 in the same browser. No idea why.

What I've tried:
- Compared my squid.conf files - nothing special.
- Googling and mailing list archives - only found common MISS problems that
don't apply to my case.
- Have even experimented removing max-age from the headers sent by the
server and only featuring public, also set Last-Modified 10 minutes in the
past - no change.
- I've only moved one low-traffic live site to the new proxy so I can test
the old one and the new with the same app server engine.
- I know of Vary:Accept-Encoding issues with testing, so I test in the same
browser, clearing the cache before the second page access. With this, I see
HITs on the old server and MISS for the new.
- Heavy logging and debugging, checking out HTTP headers too - seems okay.
The same HTTP headers yield in HITs on the old server and MISS on the new
one.

I've collected some resources:
- Old server squid.conf - http://pastebin.com/h0C7t96n
- New server squid.conf - http://pastebin.com/KaHDVWYt
- Examples that gets cached on both servers - http://pastebin.com/Be4RqVLq
 , http://pastebin.com/yVDeuyQp
- Example that gets cached on the old server but is a MISS on the new one -
http://pastebin.com/VvTU6ieR vs http://pastebin.com/ysKJwbmh
- Old server firewall settings - http://pastebin.com/v688dDMU
- New server firewall settings - http://pastebin.com/HyzGdjb4

Any tips, hints? This is driving me crazy!

Best regards,
Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150224/c8f8fe70/attachment-0001.html>