[squid-users] Significant memory leak with version 5.x (not with 4.17)

[Praveen Ponakanti]

Hi,


We are running the squid proxy for servicing outbound HTTP quests from our
network and have observed a significant memory leak with 5.x versions.
While there are several discussions about memory leaks with recent
versions, just wanted to list out what we have observed in case this is an
unknown leak.



   - The request-rate through our squid proxy currently ranges between a
   daily low of 10 rps to a daily high 125 rps.
   - About mid-way during the daily ramp in request rates, the memory usage
   of the squid proxy starts to increase by about 1-2.5 G / day before
   leveling off till the next day's ramp in reqs. 4.17 does not exhibit this
   memory leak (or at least not at anything close to this rate).
   - We are running 30 squid workers and the cache is set to deny all.
   Besides this, we have a single tcp_outgoing_address, some site specific
   ACL’s (both IP & domain acl's), and a custom access log format with a UDP
   target.
   - Both versions of squid run on similar hosts (64 cores each) and
   receive identical traffic patterns throughout the day. Version 5.3 & 5.1
   have similar rates of memory leak.


We are unable to use version 5.X in our production environment as we will
have a much higher rate of requests through the proxy later on.


Due to the nature of the memory leak, it appears something with the memory
pool management has been broken with version 5.x.


I have attempted to build squid with -with-valgrind-debug, and run it in a
test env. However valgrind appears to collect some data from the parse
config functions and then the squid proxy restarts. Valgrind no longer
reports memory leak stats afterwards.


Snips from squid-internal-mgr/info. Note the data below does not appear to
account for all the memory used by the squid process(es), which is also
reported by the squid-exporter container. Our node level stats show that
the 5.x squid is currently using up more than 28G, while the 4.17 version
is under 7G. Both instances were setup to take traffic about 10 days ago.


Thanks

Praveen


Version 5.3

——————


Memory accounted for:

Total accounted:       1215892 KB

memPoolAlloc calls: 7529400915

memPoolFree calls:  7624989161

File descriptor usage for squid:

Maximum number of file descriptors:   31457280

Largest file desc currently in use:   24962

Number of file desc currently in use: 4486

Files queued for open:                   0

Available number of file descriptors: 31452794

Reserved number of file descriptors:  3000

Store Disk files open:                   0


Version 4.17

——————


Memory accounted for:

Total accounted:        76878 KB

memPoolAlloc calls: 6217787434

memPoolFree calls:  6301483686

File descriptor usage for squid:

Maximum number of file descriptors:   31457280

Largest file desc currently in use:   20184

Number of file desc currently in use: 4419

Files queued for open:                   0

Available number of file descriptors: 31452861

Reserved number of file descriptors:  3000

Store Disk files open:                   0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20211221/429cffd2/attachment-0001.htm>