[squid-users] cache github zip repositories

[Hardik Dangar]

Amos,

Thank you very much for you reply,

But I would just like to confirm this,

I thought ETAG is something which we can verify like,

- when the server is sending response it sends ETAG.
- squid then decides to cache it with ETAG
- now if somebody tries to fetch the same URL given they are on same squid
network, squid will send previously saved copy of the ETag along with the
request in a "If-None-Match" field.
- On this subsequent request, the server may now compare the client's ETag
with the ETag for the current version of the resource. If the ETag values
match, meaning that the resource has not changed, then the server may send
back a very short response with a HTTP 304 Not Modified status. The 304
status tells the client that its cached version is still good and that it
should use that.

Now comparing this to our GitHub example.
- when we request https://codeload.github.com/hardikdangar/test/zip/master
it has
 ETag: "9ea9838812d6f7bc53763eb1577da04e2fa473d5"
- even if you request it after a day ETAG remains the same as long as
repository is not changed.
- So when i request that file again squid would not send request with ETAG
to server ?

( just to clarify, github does not change ETAG unless you change files in
repo). My concern is if ETAG is not changed then we want to use the cache
but if its change then we want to download copy of new version.

now based on your answer you have said it will not request the file again
until min time defined in referesh_pattern. what if i set the request min
time to be like 1 minute in referesh_pattern ? will it cause squid to check
ETAG with server and it will serve cached file as ETAG is not changed. or
will it just drop the cached file after a minute and download new file ?

Sorry for all of my noob questions but i am just trying to understand squid
and its options. it's really fascinating software and really appreciate
your answers here.

Thank you very much.
Have a good day.




On Fri, Sep 16, 2016 at 12:39 PM, Hardik Dangar <
hardikdangar+squid at gmail.com> wrote:

> *Amos,*
>
> Thanks for the reply but it seems i am not able to tell you what i want to do. i don't want to cache repo files. i want to cache .zip files only. i don't want .git file to cache but only .zip files which are fetch from github.com,
>
> Also you have said things about commits but i am talking about zip file which is given by github via download button or composer fetches those files via command line directly. as soon as someone commits zip file's ETag is changed when you fetch it.
>
> So there is no way to achieve this ? Do you think there is a way to achieve this ?
>
>
> *Consider: how does Squid know the ETag has changed on the server?**What you know about things happening in RL is not what Squid knows.*
>> *I fact how do *you* know someone else did not commit a change during*
>> *that ~1 second it takes to look at the page and click the download button?** Simply, you don't, and cannot until the new object has been fetched.*
>> *Likewise, Squid cannot know if the object is the same until it has*
>> *fetched a MISS from the server. Except that Squid does not look at the*
>> *previous page content, so it cannot even 'see' if there is a commit*
>> *listed there that might be different since whenever it got the previous**object.*
>> *There is no Cache-Control or Expires header indicating a specific*
>> *storage timeout or revalidation procedure. So refresh_pattern defaults*
>> *will be used. These responses will be cached for the refresh_pattern**'Min' duration (900 minutes) before being considered for revalidated.*
>> *NP 1: Synthesizing Last-Modified from the Date header is only just being*
>> *fixed in Squid the past few weeks, and some parts of it still to be*
>> *committed. So I would not expect that response to be revalidated, just**re-fetched fully in older Squid.*
>> *NP 2: The Vary header indicates that every person logged in gets a*
>> *differently cached response based on how their credentials are hashed on*
>> *each request (in Authorization tokens). So caching these objects will*
>> *not help much with many developers involved. It will be of most help for**the anonymous visitors where username is always a generic NIL value.*
>> *HTH**Amos*
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160917/cb6cc96d/attachment-0001.html>