[squid-users] Rate limiting outbound connections with http_access?

[Mike Glover]

Hi,

My project makes user-initiated requests to a selection of HTTPS API,  I'm using squid 5.7 as a forward proxy with SSL bumping to aggressively cache results, and it's working great for that.

One of the API (let's call it 'foobar.org') has a strict 1 request per second limit. I would like to throttle outbound requests from my server using squid.*

I've written a simple external ACL program (rate_limit.py) that works as a throttle, and I've hooked it up like this in my config:

acl delayhosts dstdomain foobar.org

external_acl_type rate1 ttl=0 children-max=1 children-startup=1 %ACL \
    ./rate_limit.py
acl 1ps external rate1

acl putdelay annotate_transaction needs_delay=1
acl checkdelay all-of !CONNECT delayhosts putdelay
acl getdelay note needs_delay
acl dodelay all-of getdelay 1ps

# dodelay can and should move somewhere after the cache check
http_access allow checkdelay dodelay

This is almost what I'm looking for.**  The problem is that the delay happens before the cache check, so I'm needlesslly throttling requests that I can serve locally. 

I can't find any hook post-cache-check that will accept a slow ACL.  Does such a thing exist in squid?

Best,

-mg

* Yes, perhaps this would be simpler with iptables.  I'm not currently using iptables in this project, I'm not terribly familiar with it, and everything else works happily unprivileged, so even a slightly kludgy solution in squid would be preferable (at this stage, at least) than learning, configuring, monitoring, and debugging another component.

** And yes, better than iptables rn