[squid-users] Sudden but sustained high bandwidth usage

[Eliezer Croitoru]

On 07/03/2016 22:08, Yuri Voinov wrote:
> 90 Gb first, 300 Gb second.
Thanks but...
Wouldn't it be much simpler and cheaper to just use WSUS instead all of 
the hassle??(if it's a closed business environment)
And when does the TCP_SWAPFAIL_MISS happens? always?
And a little tweak for the squid.conf
read_ahead_gap 4096 KB

The above doesn't match your environment bandwidth.
You are just spending too much bandwidth on requests that might not be 
fulfilled.
Try changing the settings to:
read_ahead_gap 128 KB

and see if it helps with something.
Also since your issue is bandwidth and users are not allowed to the 
Internet without the proxy I would try to dump the pfsense pf states to 
see what happens in the network layer, what src IP is consuming all this 
bandwidth.(or get a wider picture)
Also squid would not be the answer for a fully reasonable network usage, 
it only comes to help with couple specific things and not to mirror the 
whole Internet or even just MS as it is.(not saying that your case is a 
trial such as this)
MS has more then one API that can be mistaken as a download and it can 
consume more then actually required caching.

I do not know exactly what this means from the info page:
         Maximum number of file descriptors:   81920
         Largest file desc currently in use:   6157
         Number of file desc currently in use: 8216

If the number of FD currently in use is 8216 then the largest file desc 
currently in use doesn't match.
This specific question might be a bug or expected result but I do not 
know and Amos or Alex or others might know the answer for this specific 
info page issue.

Another question I have which might be related(I have experienced such 
issues with GlusterFS in the past), how is the VM cache disk\s are 
connected? Is it connected directly to the VM or in the hypervisor level?
How do you mount them(fstab)? are these on the root disk or do you have 
couple disks mounted?
Did you had the chance to try to use other FS then EXT4? reiserFS? XFS? 
other?

The above questions are related to the TCP_SWAPFAIL_MISS.
Since there is an issue and you are probably only on the "buffering" 
testing stage of the cache_dir I would try to somehow reproduce the 
issue but it's not clear to me what is the exact way to replicate the issue.

Thanks,
Eliezer