[squid-users] Rate limiting outbound connections with http_access?

[Alex Rousskov]

On 7/13/23 20:51, Mike Glover wrote:

> My project makes user-initiated requests to a selection of HTTPS API,  I'm using squid 5.7 as a forward proxy with SSL bumping to aggressively cache results, and it's working great for that.
> 
> One of the API (let's call it 'foobar.org') has a strict 1 request per second limit. I would like to throttle outbound requests from my server using squid.*
> 
> I've written a simple external ACL program (rate_limit.py) that works as a throttle, and I've hooked it up like this in my config:
> 
> acl delayhosts dstdomain foobar.org
> 
> external_acl_type rate1 ttl=0 children-max=1 children-startup=1 %ACL \
>      ./rate_limit.py
> acl 1ps external rate1
> 
> acl putdelay annotate_transaction needs_delay=1
> acl checkdelay all-of !CONNECT delayhosts putdelay
> acl getdelay note needs_delay
> acl dodelay all-of getdelay 1ps
> 
> # dodelay can and should move somewhere after the cache check
> http_access allow checkdelay dodelay
> 
> This is almost what I'm looking for.**  The problem is that the delay happens before the cache check, so I'm needlesslly throttling requests that I can serve locally.
> 
> I can't find any hook post-cache-check that will accept a slow ACL.  Does such a thing exist in squid?


Yes, several post-cache-check directives support slow ACLs, including 
always_direct and never_direct.

Technically, http_reply_access fits that criteria as well, but that 
directive is checked too late for your purposes AFAICT: You need a 
directive that is checked _before_ Squid sends the request.


HTH,

Alex.


> Best,
> 
> -mg
> 
> * Yes, perhaps this would be simpler with iptables.  I'm not currently using iptables in this project, I'm not terribly familiar with it, and everything else works happily unprivileged, so even a slightly kludgy solution in squid would be preferable (at this stage, at least) than learning, configuring, monitoring, and debugging another component.
> 
> ** And yes, better than iptables rn
> 
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users