[squid-users] Excessive TCP memory usage

[Amos Jeffries]

On 25/05/2016 5:50 a.m., Deniz Eren wrote:
> Hi,
> 
> After upgrading to squid 3.5.16 I realized that squid started using
> much of kernel's TCP memory.

Upgrade from which version?

> 
> When squid was running for a long time TCP memory usage is like below:
> test at test:~$ cat /proc/net/sockstat
> sockets: used *
> TCP: inuse * orphan * tw * alloc * mem 200000
> UDP: inuse * mem *
> UDPLITE: inuse *
> RAW: inuse *
> FRAG: inuse * memory *
> 
> When I restart squid the memory usage drops dramatically:

Of course it does. By restarting you just erased all of the operational
state for an unknown but large number of active network connections.

Whether many of those should have been still active or not is a
different question. the answer to which depends on how you have your
Squid configured, and what the traffic through it has been doing.


> test at test:~$ cat /proc/net/sockstat
> sockets: used *
> TCP: inuse * orphan * tw * alloc * mem 10
> UDP: inuse * mem *
> UDPLITE: inuse *
> RAW: inuse *
> FRAG: inuse * memory *
> 

The numbers you replaced with "*" are rather important for context.


> I'm using Squid 3.5.16.
> 

Please upgrade to 3.5.19. Some important issues have been resolved. Some
of them may be related to your TCP memory problem.


> When I look with "netstat" and "ss" I see lots of CLOSE_WAIT
> connections from squid to ICAP or from squid to upstream server.
> 
> Do you have any idea about this problem?

Memory use by the TCP system of your kernel has very little to do with
Squid. Number of sockets in CLOSE_WAIT does have some relation to Squid
or at least to how the traffic going through it is handled.

If you have disabled persistent connections in squid.conf then lots of
closed sockets and FD are to be expected.

If you have persistent connections enabled, then fewer closures should
happen. But some will so expectations depends on how high the traffic
load is.

Amos