[squid-users] cache github zip repositories

[Hardik Dangar]

Hello,

I am trying to cache Github zip URL's so it can be effectively cached as a
composer(php dependency management tool) uses them and in our local setup (
we are about 40 developers on a Lan and it will really help us managing
cache.). My squid version is 3.5.12 and our squid cache server is ubuntu
16.04. Here is squid.conf file we use,
https://gist.github.com/hardikdangar/df31d5bce725eff66e06f3abd6e77600

Here is the part which I want to cache,
say for example you want to download repo from GitHub then URL looks like
https://github.com/hardikdangar/test/archive/master.zip
but it redirects to the following,
https://codeload.github.com/hardikdangar/test/zip/master

You can see the response parameters via redbot.org
https://redbot.org/?uri=https%3A%2F%2Fcodeload.github.com%
2Fhardikdangar%2Ftest%2Fzip%2Fmaster

  HTTP/1.1 200 OK
    Content-Length: 929
    Access-Control-Allow-Origin: https://render.githubusercontent.com
    Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'
    Strict-Transport-Security: max-age=31536000
    Vary: Authorization,Accept-Encoding
    X-Content-Type-Options: nosniff
    X-Frame-Options: deny
    X-XSS-Protection: 1; mode=block
    ETag: "9ea9838812d6f7bc53763eb1577da04e2fa473d5"
    Content-Type: application/zip
    Content-Disposition: attachment; filename=test-master.zip
    X-Geo-Block-List:
    Date: Wed, 14 Sep 2016 23:24:44 GMT
    X-GitHub-Request-Id: 77092BF1:7F40:346461:57D9DC3C

Now if i do any change to above repository github does change ETAG and if i
don't change anything then ETAG remains the same so i believe we should be
able to cache those .zip files.

By default, squid does not cache codeload.github.com, to put it into cache,
I added,
refresh_pattern codeload.github.com 900 20% 4320 reload-into-ims

Now as per my understanding this should check etag as Last-Modified is not
provided by github for each new request. This does cache the zip file but
what happens is in next request even if i change the content and etag
changes squid sends the cached file from its cache instead of downloading
new file.

I have no clue why this happens. Can anyone help me figure out what's wrong
here? why squid does not detect new etag when repository is updated? why it
sends cache file even though there is new file available.

Thank you very much in advance for reading upto this point and have a good
day.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160915/3acc26a2/attachment.html>