[squid-users] url_rewrite_program and ACLs

[Vieri]

________________________________
From: Amos Jeffries <squid3 at treenet.co.nz>
>
> I would compare your custom script to the ext_sql_session_acl.pl.in 
> script we bundle with current Squid.
>  If yours lacks concurrency channel-ID I highly recommend adding that 
> behaviour.
>  If the DB is designed to store the protocol scheme, domain[:port] and 
> path?query portions of URLs in separate columns it will be more 
> efficient to pass those parameters as separate (%PROTO %DST %PORT %PATH) 
> to the helper instead of just %URI.
>
> The overheads in Squid of using external_acl_type helper interface 
> should be slightly less than the url_rewrite_program for SG. The SQL DB 
> data loading is about the same or better than what SG does AFAIK.


Thanks for the info. The script I was running was actually using concurrency with channel IDs. I also think it was correctly closing all file handles, DB connections, etc. However, I'm now merging your script which looks tidier into the one I was using. I'll see how it behaves over a few days.


> Ouch, but kind of expected with those FD number increase.

I'll have to find out why I'm still seeing this number rise albeit slower.
> Which reminds me ... Are you using SSL-Bump? if so ensure that you have 
> configured "sslflags=NO_DEFAULT_CA" on the port lines. The default 
> Trusted CA set can add a huge amount of useless memory to each client 
> connection, which can add up to many GB quite quickly.


Many thanks. Applied. I also noticed that Squid 4 defaults to tls_default_ca=off. Will keep that in mind when migrating to v.4 (actually, I'll just need to remove sslflags=NO_DEFAULT_CA).

> Nod, until the RAM runs out entirely, then problems are definitely to be 
> expected and that sounds like it is your problem now.


I really don't know how Linux manages memory, but despite my open FDs growing steadily and my "free" RAM slowly decreasing, at some point I noticed that the FDs kept growing slowly while the free mem suddenly went back up a bit (not a whole lot, but significantly -- around 0.5-0.7GB sudden increase).
> I actually physically blew up a test 
> machine (fire and smoke puring out the back!) measuring the effects of 
> RAID on a overloaded proxy about a decade ago.


I don't need this kind of horror stories... :-) Not yet at least. Let me first get a grip on my installation.

Thanks,

Vieri