[squid-users] Sudden but sustained high bandwidth usage

Mon Mar 7 22:08:21 UTC 2016

I know what is happening. I just don't know how to fix it without 
breaking windows updates caching.
The "extra" traffic is coming from windows updates mirrors.

/acl wupdatecachable url_regex -i 
(microsoft|windowsupdate)\.com.*\.(cab|exe|ms[i|u|f]|dat|zip|psf|appx|appxbundle|esd)//
//range_offset_limit none wupdatecachable/

Commenting this second line is enough to put an end to the insane 
downloads. But then all the RANGE requests of windows updates won't be 
cached.

I don't know how to explain these FD numbers. I'm using EXT4 and I don't 
know what are vmware cache disks.

Oh, and it seems not only windows updates' range requests are giving 
trouble:

1457381845.725    241 10.101.1.23 TCP_SWAPFAIL_MISS/206 33123 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.036    293 10.101.1.23 TCP_SWAPFAIL_MISS/206 151352 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.120     66 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.310     66 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.397     60 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.514     63 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.633     65 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.747     63 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.839     63 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381846.949     81 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381847.041     61 10.101.1.23 TCP_SWAPFAIL_MISS/206 33129 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381847.192    118 10.101.1.23 TCP_SWAPFAIL_MISS/206 65897 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381847.447    207 10.101.1.23 TCP_SWAPFAIL_MISS/206 121720 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2015-10/1043520/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381862.824    639 10.101.1.23 TCP_SWAPFAIL_MISS/206 33122 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381865.720   2879 10.101.1.23 TCP_SWAPFAIL_MISS/206 221234 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381866.078    343 10.101.1.23 TCP_SWAPFAIL_MISS/206 33130 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381867.107    995 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381867.523    400 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381867.830    291 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381868.082    238 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381868.318    219 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381868.503    171 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381868.695    174 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381868.880    170 10.101.1.23 TCP_SWAPFAIL_MISS/206 33128 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381869.179    286 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381869.424    231 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381869.761    322 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381870.008    232 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381870.236    214 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381870.429    178 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381870.609    174 10.101.1.23 TCP_SWAPFAIL_MISS/206 65896 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381870.805    180 10.101.1.23 TCP_SWAPFAIL_MISS/206 65897 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf
1457381871.008    189 10.101.1.23 TCP_SWAPFAIL_MISS/206 65898 GET 
http://www.stf.jus.br/arquivo/biblioteca/NovasAquisicoes/2016-02/1022313/sumario.pdf 
- HIER_DIRECT/201.49.144.135 application/pdf

Best Regards,

-- 
Heiler Bemerguy - (91) 98151-4894
Assessor Técnico - CINBESA (91) 3184-1751

Em 07/03/2016 18:00, Eliezer Croitoru escreveu:
> On 07/03/2016 22:08, Yuri Voinov wrote:
>> 90 Gb first, 300 Gb second.
> Thanks but...
> Wouldn't it be much simpler and cheaper to just use WSUS instead all 
> of the hassle??(if it's a closed business environment)
> And when does the TCP_SWAPFAIL_MISS happens? always?
> And a little tweak for the squid.conf
> read_ahead_gap 4096 KB
>
> The above doesn't match your environment bandwidth.
> You are just spending too much bandwidth on requests that might not be 
> fulfilled.
> Try changing the settings to:
> read_ahead_gap 128 KB
>
> and see if it helps with something.
> Also since your issue is bandwidth and users are not allowed to the 
> Internet without the proxy I would try to dump the pfsense pf states 
> to see what happens in the network layer, what src IP is consuming all 
> this bandwidth.(or get a wider picture)
> Also squid would not be the answer for a fully reasonable network 
> usage, it only comes to help with couple specific things and not to 
> mirror the whole Internet or even just MS as it is.(not saying that 
> your case is a trial such as this)
> MS has more then one API that can be mistaken as a download and it can 
> consume more then actually required caching.
>
> I do not know exactly what this means from the info page:
>         Maximum number of file descriptors:   81920
>         Largest file desc currently in use:   6157
>         Number of file desc currently in use: 8216
>
> If the number of FD currently in use is 8216 then the largest file 
> desc currently in use doesn't match.
> This specific question might be a bug or expected result but I do not 
> know and Amos or Alex or others might know the answer for this 
> specific info page issue.
>
> Another question I have which might be related(I have experienced such 
> issues with GlusterFS in the past), how is the VM cache disk\s are 
> connected? Is it connected directly to the VM or in the hypervisor level?
> How do you mount them(fstab)? are these on the root disk or do you 
> have couple disks mounted?
> Did you had the chance to try to use other FS then EXT4? reiserFS? 
> XFS? other?
>
> The above questions are related to the TCP_SWAPFAIL_MISS.
> Since there is an issue and you are probably only on the "buffering" 
> testing stage of the cache_dir I would try to somehow reproduce the 
> issue but it's not clear to me what is the exact way to replicate the 
> issue.
>
> Thanks,
> Eliezer

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160307/3ecfcab9/attachment-0001.html>