[squid-users] Never expire any object Squid configuration

[Amos Jeffries]

On 20/04/2016 7:24 p.m., Veiko Kukk wrote:
> Hi,
> 
> We have a Squid between our server application and openstack swift
> backend in accel/reverse mode with store-id configuraton (to strip
> temporary authentication URL-s). We want that any object that has been
> stored in squid cache is never again fetched from source and never again
> checked if it is fresh. Well, never in this case could be one year.

The usual resonse to this is "Squid is a cache, not an archive".

Squid obeys the instructions given to it by the client and the server.
If the union of those instructions requires a fetch or revalidation it
is important to do so to ensure data integrity.

> 
> Relevant section from current configuration:
> 
> refresh_pattern -i ^https:\/\/AUTH_.*squid.internal.* 526000 100% 526000
> override-expire ignore-reload ignore-no-store ignore-private store-stale
> max-stale=52
> 
> With this configuration, we still see lots of TCP_REFRESH_MODIFIED/200
> TCP_REFRESH_UNMODIFIED/200 and TCP_REFRESH_UNMODIFIED/304 in logs.
> 
> How must Squid be configured to completely disable any refreshes? Those
> objects never change after they have been created and we only want them
> to be pushed out from cache by cache replacement policy.


The refresh_pattern ignore-* and override-* are hacks for ISP proxies to
cope with and fix traffic from badly designed web applications. They do
not have control over the origin so at times have no other choice.

A good rule of thumb to follow with reverse-proxy is that any problem
you see with your proxy, so does every other proxy on the planet have
with your sites traffic. It is best to fix things using HTTP mechanisms
properly, not with proxy config workarounds. That way you can benefit
from those other proxies caching as well as your own.


The correct way to do this in HTTP is to have the origin server /
application deliver an Expires: header with a date 1 year in the future
and/or a Cache-Control header with max-age=N with an equivalent value in
seconds. Squid will obey that.

 NP: HTTP allows caching for up to 68 years (2^31 seconds). Squid will
revalidate objects older than 1 year.

If you want objects to be cached, *do not* send Cache-Control:no-store
or Cache-Control:private on them. Nor small values of max-age=N.

If you want to avoid revalidation, *do not* send Cache-Control with
must-revalidate, proxy-revalidate, no-cache, or private. Also avoid
www-auth.


A reverse-proxy has some more options than other proxies:

* As a reverse-proxy you are allowed to use the 'ignore-cc' option on
http_port directives. This will cause the reverse-proxy to ignore any
client Cache-Control demands for revalidation.

* reverse-proxy contain the Surrogate feature. This can be used to give
the CDN proxy completely different max-age values than are sent in
Cache-Control for general caching.
 <http://www.squid-cache.org/Doc/config/http_accel_surrogate_remote/>
 <http://www.squid-cache.org/Doc/config/httpd_accel_surrogate_id/>


> 
> I read from old post
> http://www.squid-cache.org/mail-archive/squid-dev/201108/0029.html that
> if client requests object without etag and server sends with etag, then
> Squid fetches object again. How to disable this?

What I was trying to convey in that post was that empty / non-existent
is one of the valid ETag values. ETag is a unique identifier within the
URI scope. If the client explicitly asks for a variant of the object in
a conditional If-Match request, then using another value of ETag to
respond is not valid/legal.

Think of ETag like a MD5/SHA hash of the object. You cant just decide
that the value is different - you have to actually change the object to
make it different. Then you have two versions of it with different tags
and the client might ask specifically about either one.

Amos