[squid-users] Sudden but sustained high bandwidth usage

[Eliezer Croitoru]

Sorry about the confusion\misunderstanding.. my brains cache is kind of 
tiny\short and I am not sure but was it you that asked about the big 
NETAPP cache a question not long ago? or was it someone else? I am maybe 
confusing because the other one had more clients but a similar issue.

I will later read and try to understand the info page again but I just 
wanted to clear something out about storage which might be known in some 
places but not to everybody.

Every storage system requires some logical and physical layer and each 
and every one of them is doing something to any IO that happens!!!
I do not say that there is an issue with the raid or the storage but it 
is clear that the storage else then the RAID needs some kind of "cache" 
or some level of buffering in order to work better. It's the same for 
DAS, SAN, NAS and any other form of storage. This is the design of such 
products. The only place which a storage is almost always directly 
accessed by most IO calls is the RAM and in the DAS area it's SSD and 
RAM+battery based products which "cache" is only slowing down the RAM 
and CPU.

I am not sure I interpret the cache_dir the docs state:
http://west.squid-cache.org/Doc/config/cache_dir/
	    cache_dir rock Directory-Name Mbytes [options]

which means that you are using:
cache_dir rock /cache2/rock1 90000 min-size=0 max-size=32768
cache_dir rock /cache/rock1 300000 min-size=32769 max-size=10737418240

90K MB on the first?
300K MB o the second? right?

How much is it in GB??

Eliezer

On 07/03/2016 20:38, Heiler Bemerguy wrote:
> skyrocketing = using our maximum link download bandwidth.
> This machine is only proxying. Not being a firewall, not a router, nor a
> gateway. It has access to the internet through our gateway/firewall
> (pfsense).
> Lots of LAN clients are connected to the proxy, this is their only way
> to the internet. 1 interface, debian linux. EXT4 FS. CPU/MEM usage is
> always stable.
> Clients use it explicitly or via wpad. Never transparently. Now I'm
> using 3 workers, because 1 is not enough and we have spare cores.
> It's a VM machine with netapp storage. lots of raid disks.
> SQUID was running perfectly without cache_dirs.
>
> I think squid is downloading and redownloading the same files over and
> over again because: 1- these are segmented downloads and
> range_offset_limit is set to NONE for these files. 2- it can't store the
> downloaded files on the cache but I don't know why!