[squid-users] Squid 3.5.7, cache_swap_high, bug or not bug ?

[Amos Jeffries]

On 21/08/2015 2:26 a.m., Stakres wrote:
> Hi Amos,
> 
> Any update ?
> 

Spent the afternoon looking into it. There is now a patch in the bug
report with some experimental changes.

> This morning, it was crazy:
> *Percent used: 100.76%*
> How is it possible ?

It seems all that documentation about the watermarks I was reading was
b*sh*t. :-(

The actual code is purging no more than 80 objects per second. And like
the bug said, only gets that many if its actually over 100% filled already.

> 
> Then the squid has crashed and it's now cleaning objects.
> I can understand new objects could be added faster than the squid is able to
> clean older objects, but it seems there is something wrong in the cleaning
> process.
> The wiki says "/As swap utilization gets close to high-water mark object
> eviction becomes more aggressive./". From my point of view the "aggressive"
> is not aggressive enough... 
> 
> Possible to have a patch/workaround soon ?

Yeah 80 remove/sec is *way* to small for a proxy that can and probably
is handling ~2K add/sec.


In that experimental patch I've done a few things that you can help test
out:

* tripled the basic removal rate to 200/sec

* set the rate of purging to reach maximum speed at high watermark (as
previously documented).

- between those two adjustments is it fast enough? all it has to do is
counter the completely new additions rate. general traffic req/sec doing
replace on content does not matter.


* removed the limit on purges when cache is >100% full, it just keeps
going until under-100% is reached or it cant go further. Squid service
is effectively paused until the cache is back within its size limit.
That may not be the best option, but its doable without a lot of change
all over the store code. Better tuning on the base removal rate should
avoid it happening in the first place.

- is that pause acceptibly small?


- do all these changes actually help in your situation any?

BTW, "debug_options 47,3" to get the purging event debug traces.

Amos