[squid-users] Squid high memory usage

[Marcus Kool]

On 06/06/2016 07:27 AM, FredB wrote:
> Thanks for your answer
>
>> What is cache_mem ?
>> See also http://wiki.squid-cache.org/SquidFaq/SquidMemory
>>
>
> Actually 25 Gb
> I tried different values, but I guess no matter, the problem is that the squid limit is only 50% of ram
>
>>> After that the swap is totally full and kswap process gone mad ...
>>> I tried with vm.swappiness = 0 but same problem, perhaps a little
>>> better, I also tried memory_pool off without any change.
>>
>> I recommend vm.swappiness = 5 to have 5% of the memory be used for
>> the file system cache and maintain good disk I/O.
>
> More I increase vm.swappiness more I swap and more I have problems, but I will try your value
>
>>
>> The values are too high (1024 times).  I think that you incorrectly
>> set cache_mem.

ah, I misread the values. I interpreted the comma as a thousand separator.

>> Start with setting  cache_mem to 16 GB
>>
>
> Maybe I misunderstand your point, but when I reduce cache_mem yes there is no problem but Squid uses only 20/30 Go Max

As you can read in the memory FAQ, the value of cache_mem is a part (often 35%) of total memory use.
When you start with a clean install, you will see a "low memory use" since the disk cache is not yet fully populated
and perhaps because the number of connections is low.

> With cache_men 15 Gb squid eats 36 % of memory
> Htop and htops reports 30 Go of free memory
>
> free -h
>               total       used       free     shared    buffers     cached
> Mem:           63G        62G       425M       122M       1,7G        27G
> -/+ buffers/cache:        33G        30G
> Swap:         1,9G       102M       1,8G
>
> All my RAM is consumed by cache/buffers and seems not be freed when it is needed by Squid

This is a healthy start.
When your disk cache is fully populated and when there is still room (e.g. 'cached' column of free shows many gigabytes), you may increase cache_mem.
Note that connections and Squid buffers occupy memory so always be a bit conservative to prevent swapping.

2 GB swap for a 64 GB memory system is a bit small. If you only have Squid and no other applications on this system it may be sufficient.

Marcus