[squid-users] client_delay_pools doesn't work as expected

[Ivan Larionov]

Hello.

We've recently had an incident where misbehaving cluster of clients started
fetching 4MB file from squid cache with ~1200 RPS (slowed down to 600 RPS
later) which resulted in up to 2Gb/s of traffic sent to clients from each
of our squid hosts and quickly overloaded squid.

I'm trying to use client_delay_pools to limit bandwidth per client and
prevent misbehaving actors from saturating client-side network / CPU on
squid hosts.

However I can't get it to work reliably. It seems to be working as expected
for cache MISS, e.g. getting a speed limit of 10MB/s. But it's completely
broken for cache HIT, speed I'm getting is ~5KB/s!

The following configuration:

client_delay_pools 1
client_delay_access 1 allow localnet
client_delay_access 1 deny all
client_delay_parameters 1 10000000 20000000

Testing with an already cached big object (2GB ISO file).

client_delay_pools disabled MISS: 20MB/s (probably speed limit on origin
side)
client_delay_pools disabled HIT: 110MB/s (probably EBS disk speed)

client_delay_pools enabled MISS: 10MB/s (limit from client_delay_parameters)
client_delay_pools enabled HIT: 5KB/s (what ???)

I retested with a smaller file (337MB) but it made no difference. Still got
5KB download speed on cache HIT.

Any ideas? Am I doing something wrong? Any other ways to limit client-side
bandwidth?

Squid version:

Squid Cache: Version 4.14
Service Name: squid
configure options:  '--program-prefix=' '--prefix=/usr'
'--exec-prefix=/usr' '--bindir=/usr/sbin' '--sbindir=/usr/sbin'
'--sysconfdir=/etc/squid' '--libdir=/usr/lib' '--libexecdir=/usr/lib/squid'
'--includedir=/usr/include' '--datadir=/usr/share/squid'
'--sharedstatedir=/usr/com' '--localstatedir=/var'
'--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-epoll'
'--enable-removal-policies=heap,lru' '--enable-storeio=aufs,rock'
'--enable-delay-pools' '--with-pthreads' '--enable-cache-digests'
'--with-large-files' '--with-maxfd=16384' '--enable-htcp'

-- 
With best regards, Ivan Larionov.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210331/ebbe59bf/attachment.htm>