[squid-users] WARNING: Disk space over limit

[Amos Jeffries]

On 12/07/17 10:47, Yuri wrote:
> 
> Well. Let's read squid.conf.documented:
> 
> #  TAG: cache_swap_low    (percent, 0-100)
> #    The low-water mark for AUFS/UFS/diskd cache object eviction by
> #    the cache_replacement_policy algorithm.
> #
> #    Removal begins when the swap (disk) usage of a cache_dir is
> #    above this low-water mark and attempts to maintain utilization
> #    near the low-water mark.
> #
> #    As swap utilization increases towards the high-water mark set
> #    by cache_swap_high object eviction becomes more agressive.
> #
> #    The value difference in percentages between low- and high-water
> #    marks represent an eviction rate of 300 objects per second and
> #    the rate continues to scale in agressiveness by multiples of
> #    this above the high-water mark.
> #
> #    Defaults are 90% and 95%. If you have a large cache, 5% could be
> #    hundreds of MB. If this is the case you may wish to set these
> #    numbers closer together.
> #
> #    See also cache_swap_high and cache_replacement_policy
> 
> Eviction starts when you cache achieves cache_swap_low. So, may be you
> traffic is more than squid can be replaced between 90 and 95%. So, you
> can try to decrease cache_swap_low. Or, review your caching policy to be
> less aggressive. Or, increase cache_dir limit. Or all together.
> 

Or adjust the maximum object size that cache can store.

Since the warnings are clearly decreasing in overage value things seem 
to be recovering properly. The warning is to alert you that it happened 
in the first place, so traffic may need re-evaluation. Your proxy seems 
to be able to evict ~400 MB/cycle from disk.


Additional to traffic rate, it is rare but can happen that one or more 
very large objects are fetched very close together. Squid then discovers 
it has to throw away GB of objects to make room for them, even though 
they may not have fully arrived yet. This may exceed the eviction rate 
for a while, depending on your disk I/O speed.




The math if you are interested:

At your total cache size, the 5% high-low gap is only 750 MB, and the 
warning occurs at 100% - just another 750 MB above that gap. Giving a 
total burst size of 1.5 GB of object(s) necessary to trigger it.

NP: that 1.5GB needs to be above the regular eviction rate per garbage 
cycle. If you are having trouble the fix is to a) get faster RPM disks, 
b) spread the load over more proxies,


I notice that the overage value in the trace is only 2.2 GB which is 
within realm for DVD images, HD movies, Windows 10 updates, etc.
If you are intending to cache those you need; multiple cache disks, 
lower high/low percentages.

  You can also ensure the physical disk has a sufficient buffer of space 
to not worry about occasional of these warnings. That is effectivively 
the same as lowering the low watermark but prohibits Squid from using 
that space when traffic is not being such a problem.


Amos