[squid-users] url_rewrite_program and ACLs

[Vieri]

________________________________
From: Amos Jeffries <squid3 at treenet.co.nz>
>
> I would compare your custom script to the ext_sql_session_acl.pl.in 
> script we bundle with current Squid.


I've rewritten my perl script, and have been running it for a full week now without any issues. Free RAM drops down to alarming values, but then rises back up again. In any case, "used swap" is always the same. The only thing that keeps be edgy is the fact that the open FDs keep growing (slowly but steadily). After a few days the value is around 6000, but after a week (today) it's:

Squid Object Cache: Version 3.5.27-20171101-re69e56c
Build Info:
Service Name: squid
Start Time:     Mon, 13 Nov 2017 11:06:36 GMT
Current Time:   Mon, 20 Nov 2017 08:48:00 GMT
Connection information for squid:
Number of clients accessing cache:      582
Number of HTTP requests received:       6435251
Number of ICP messages received:        0
Number of ICP messages sent:    0
Number of queued ICP replies:   0
Number of HTCP messages received:       0
Number of HTCP messages sent:   0
Request failure ratio:   0.00
Average HTTP requests per minute since start:   647.3
Average ICP messages per minute since start:    0.0
Select loop called: 246503925 times, 2.420 ms avg
Cache information for squid:
Hits as % of all requests:      5min: 4.4%, 60min: 4.3%
Hits as % of bytes sent:        5min: -0.7%, 60min: -6.0%
Memory hits as % of hit requests:       5min: 75.4%, 60min: 67.9%
Disk hits as % of hit requests: 5min: 0.0%, 60min: 0.1%
Storage Swap size:      29848 KB
Storage Swap capacity:  91.1% used,  8.9% free
Storage Mem size:       29120 KB
Storage Mem capacity:   88.9% used, 11.1% free
Mean Object Size:       13.19 KB
Requests given to unlinkd:      97921
Median Service Times (seconds)  5 min    60 min:
HTTP Requests (All):   0.18699  0.19742
Cache Misses:          0.19742  0.20843
Cache Hits:            0.00000  0.00000
Near Hits:             0.00000  0.27332
Not-Modified Replies:  0.00000  0.00000
DNS Lookups:           0.08334  0.07618
ICP Queries:           0.00000  0.00000
Resource usage for squid:
UP Time:        596484.490 seconds
CPU Time:       15823.550 seconds
CPU Usage:      2.65%
CPU Usage, 5 minute avg:        4.38%
CPU Usage, 60 minute avg:       4.86%
Maximum Resident Size: 14493888 KB
Page faults with physical i/o: 0
Memory accounted for:
Total accounted:       -862888 KB
memPoolAlloc calls: 2199430697
memPoolFree calls:  2241183896
File descriptor usage for squid:
Maximum number of file descriptors:   65536
Largest file desc currently in use:   12714
Number of file desc currently in use: 11998
Files queued for open:                   0
Available number of file descriptors: 53538
Reserved number of file descriptors:   100
Store Disk files open:                   0
Internal Data Structures:
2520 StoreEntries
2519 StoreEntries with MemObjects
2314 Hot Object Cache Items
2263 on-disk objects


mgr:filedescriptors shows a great deal of these:

Remote Address        Description
--------------------- ------------------------------
127.0.0.1:1344        127.0.0.1


# squidclient mgr:filedescriptors | grep -c "127.0.0.1:1344"
11578


Port 1344 is where the c-icap daemon listens on.
This is the relevant part in squid.conf:

icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_encode off
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024
icap_service squidclamav respmod_precache bypass=0 icap://127.0.0.1:1344/clamav
adaptation_access squidclamav allow all
icap_service_failure_limit -1


The number of connections to this port fluctuates over time (it also decreases), but overall it clearly increases day by day.
I could have an issue with either c-icap itself or one of its modules.
I'll keep an eye on it.

Thanks,

Vieri