[squid-users] url_rewrite_program and ACLs

[Vieri]

________________________________
From: Amos Jeffries <squid3 at treenet.co.nz>
>
> Darn. You have the one case that calls for keeping the helper :-(
>
> You can still move the ACLs that load in a reasonable times into 
> squid.conf and leave the others in SG/ufdbguard. Using 
> url_rewrite_access to restrict which transactions the helper gets 
> involved with. That will reduce its latency impact on lie traffic, but 
> still cause much the same memory related (non-)issues as now.


That's exactly what I'm doing right now...
Thanks.

> Running "squid -k shutdown" a _second_ time sends the running proxy a 
> signal to immediately skip to the processing as if the shutdown_lifetime 
> had already been reached.


Thanks for that double-shutdown signal trick. I'll have to try that asap.

I'm making progress (sort of) on the FD (non-)issues I'm having.

I'll try to post back to Alex asap.

I have a custom perl script that does MySQL lookups for blacklisted sites (lots of them - so I can't use ACLs within squid.conf). I define that helper with external_acl_type.

Yesterday I changed my squid.conf by disabling this helper, and used squidGuard instead.
I noticed a huge improvement.

I took this snapshot yesterday:

15:25 08/11/2017:

File descriptor usage for squid:
Maximum number of file descriptors: 65536
Largest file desc currently in use: 2730
Number of file desc currently in use: 1838
Files queued for open: 0
Available number of file descriptors: 63698
Reserved number of file descriptors: 100
Store Disk files open: 0

Today I took another peak and found:

Thu Nov 9 12:19:05 CET 2017:

File descriptor usage for squid:
Maximum number of file descriptors: 65536
Largest file desc currently in use: 6980
Number of file desc currently in use: 6627
Files queued for open: 0
Available number of file descriptors: 58909
Reserved number of file descriptors: 100
Store Disk files open: 0

The FDs are still increasing steadily, but a LOT less.

On the other hand, the "free" RAM went from 2GB yesterday to just 275MB today:

# free --mega
total used free shared buff/cache available
Mem: 32865 8685 275 157 23904 23683
Swap: 37036 286 36750

Used swap is still low enough (unchanged actually), so I guess I don't need to worry about it.

However, I'm bound to have issues when the "free" mem reaches 0... and I bet it will eventually.
That's when the double-shutdown trick will kick in.

I'll review the perl helper code, or maybe just switch to ufdbGuard.

Thanks,

Vieri