[squid-users] caching store increase then decrease during caching windows updates and all request are TCP_MISS ??!!!

[Amos Jeffries]

On 12/09/2016 9:47 p.m., --Ahmad-- wrote:
> hi 
> 
> i have caching store increase then decrease  during caching windows updates and all request are TCP_MISS ??!!!
> 
> root at raspberrypi:~# du -sh /var/cache/squid/
> 558M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 561M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 564M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 647M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 650M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 652M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 604M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 605M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 605M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 606M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 606M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 595M    /var/cache/squid/
> root at raspberrypi:~# du -sh /var/cache/squid/
> 

What relevance is this?


> 
> here is sample of logs :
> 
> 1473672787.641    444 192.168.0.10 TCP_CLIENT_REFRESH_MISS/200 872 HEAD http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/ee82a2cb-f2cf-4cb3-907a-4f5496d2dc56? - ORIGINAL_DST/13.107.4.50 application/octet-stream

HEAD request. None of the object got downloaded.

> 1473672799.084  11876 192.168.0.10 TCP_MISS/206 615396 GET http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/ee82a2cb-f2cf-4cb3-907a-4f5496d2dc56? - ORIGINAL_DST/13.107.4.50 application/octet-stream
> 1473672808.553   8396 192.168.0.10 TCP_MISS/206 615396 GET http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/ee82a2cb-f2cf-4cb3-907a-4f5496d2dc56? - ORIGINAL_DST/13.107.4.50 application/octet-stream
> 1473672811.054  24905 192.168.0.10 TCP_MISS/206 1017331 GET http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/1fc27b6e-d563-415b-b11a-9e7e6f1ad33c? - ORIGINAL_DST/13.107.4.50 application/octet-stream

Some Range requests for URI with ?query parameters. You need to add
"strip_query_terms no" to your squid.conf. Then you will be able to see
if these are identical URI or not.

Right now all that can be said is the script on the server which is
producing objects is the same one for many of the requests. There is no
way to know if they are requesting the same object or different ones.


> 1473672812.922   2687 192.168.0.10 TCP_CLIENT_REFRESH_MISS/206 1049501 GET http://tlu.dl.delivery.mp.microsoft.com/filestreamingservice/files/ee82a2cb-f2cf-4cb3-907a-4f5496d2dc56? - ORIGINAL_DST/13.107.4.50 application/octet-stream

A request where the client required new content to be fetched.

The rest of your log shows a lot of repeats of the above things happening.


> 
> #refresh_pattern ([^.]+.|)(download|(windows|)update|).(microsoft.|)com/.*.(cab|exe|msi|msp) 4320 100% 43200 reload-into-ims
> refresh_pattern ([^.]+.)?(download|(windows)?update).(microsoft.)?com/.*.(cab|exe|msi|msp|psf) 4320 100% 43200 reload-into-ims
> refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*.(pkg|dmg) 4320 100% 43200 reload-into-ims

These refresh_pattern contain broken regex patterns.


1) Unless you explicitly use start/end anchors regex have implicit .*
sequences before and after the provided pattern.

 That means "([^.]+.|)" and "([^.]+.)?" are just a very complex (and
slow to compare) ways to write .*


2) '.' is a reserved character. If you want it to match a '.' in the
domain name you need to write it as \.


Hint both above regex will match the non-Microsoft, non-Apple URI:
  http://download.com/?pkg=caddy


Amos