[squid-users] MS Update

[Amos Jeffries]

On 11/01/2016 2:33 p.m., Alex Samad wrote:
> Hi
> 
> I burnt up 172G of download in 24 hours with multi machines doing the
> download of the same file (MS SQL patch)
> 
> I think I am running into the same issue
> 
> 
> So multiple machines are trying to do the download...
> Q) why don't they share the same download !
> 
> 1452459804.945  64052 10.172.208.108 TCP_MISS/206 1727799 GET
> http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/svpk/2015/05/sqlserver2014sp1-kb3058865-x64-enu_2c84e2ebd0d3cb4980a3a1a80d79fd7520405626.exe
> - HIER_DIRECT/150.101.195.217 application/octet-stream
> 1452459868.272  63326 10.172.208.108 TCP_MISS/206 1312208 GET
> http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/svpk/2015/05/sqlserver2014sp1-kb3058865-x64-enu_2c84e2ebd0d3cb4980a3a1a80d79fd7520405626.exe
> - HIER_DIRECT/150.101.195.217 application/octet-stream
> 1452459933.336  65061 10.172.208.108 TCP_MISS/206 1155440 GET
> http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/svpk/2015/05/sqlserver2014sp1-kb3058865-x64-enu_2c84e2ebd0d3cb4980a3a1a80d79fd7520405626.exe
> - HIER_DIRECT/150.101.195.217 application/octet-stream
> 1452459998.406  65067 10.172.208.108 TCP_MISS/206 1022158 GET
> http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/svpk/2015/05/sqlserver2014sp1-kb3058865-x64-enu_2c84e2ebd0d3cb4980a3a1a80d79fd7520405626.exe
> - HIER_DIRECT/150.101.195.217 application/octet-stream
> 1452460066.455  68046 10.172.208.108 TCP_MISS/206 2006058 GET
> http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/svpk/2015/05/sqlserver2014sp1-kb3058865-x64-enu_2c84e2ebd0d3cb4980a3a1a80d79fd7520405626.exe
> - HIER_DIRECT/150.101.195.200 application/octet-stream
> 1452460134.536  68078 10.172.208.108 TCP_MISS/206 1575462 GET
> http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/svpk/2015/05/sqlserver2014sp1-kb3058865-x64-enu_2c84e2ebd0d3cb4980a3a1a80d79fd7520405626.exe
> - HIER_DIRECT/150.101.195.200 application/octet-stream
> 1452460204.180  69643 10.172.208.108 TCP_MISS/206 1387948 GET
> http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/svpk/2015/05/sqlserver2014sp1-kb3058865-x64-enu_2c84e2ebd0d3cb4980a3a1a80d79fd7520405626.exe
> - HIER_DIRECT/150.101.195.217 application/octet-stream
> 
> 
> here you can see multiple requests for the same file .

A file which is 670 MB big.

> 
> I am presuming 206 is a partial download - is that Windows or SQUID ..

206 is a Range reply.

> I presume windows client

Presumably the client made a Range request. WU tends to do that for
large objects, and the repeated nature of the 206 with small parts of it
is a relatively strong indicator that is going on. So ...

> 
> So is it the byte range that gets cached.
> 

Squid does not cache byte ranges.

> if client a want 100 - 200 of file X
> and client B wants 50 - 150.. will squid reuse whatever has been
> downloaded of the 100-200 request by client B
> 
> 
> any way I can for the requests to a single file - I could manually
> download the file once, that would place it in the cache.
> 

Exactly as the special notice at the end of section #1 in the
SquidFaq/WindowsUpdate wiki page says. SP and similar huge updates (this
one included) need to be treated specially.

> 
> I have this in my config
> # http://wiki.squid-cache.org/SquidFaq/WindowsUpdate
> range_offset_limit 200 MB
> maximum_object_size 200 MB
> quick_abort_min -1
> 
> refresh_pattern -i
> microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80%
> 43200 reload-into-ims
> refresh_pattern -i
> windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320
> 80% 43200 reload-into-ims
> refresh_pattern -i
> windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80%
> 43200 reload-into-ims
> 
> 
> guessing I have to bump up the 200M max to 800mb.

Maybe. But IMHO use the ACLs tat range_offset_limit can take.

> are the other values still okay ?

Yes.

Amos