[squid-users] leaking memory in squid 3.4.8 and 3.4.7.

[Steve Hill]

On 30.09.14 15:13, Amos Jeffries wrote:

> IIRC the valgrind report is strangely at the end of mgr:info rather
> than mgr:mem.

In that case the wiki is wrong. :)
Anyway, it doesn't show up on either of them for me so I guess you might 
be right that it isn't "real" leaked memory.  (I still thought I'd see 
some indication that it was invoking the valgrind report though, even if 
it contained no data)

> With one small exception all the reports of memory "leaks" in
> 3.2/3.3/3.4 series are in fact memory over-allocations. The difference
> is that over-allocated memory is still referenced by some part of
> Squid, so it *does not* show up in a leak finder like valgrind.

Do you have any advice for finding out what memory is still referenced 
in large amounts?  Since this is unaccounted memory, it presumably 
doesn't appear in mgr:mem (at least, I can't see anything obvious in there)?

I'm trying to figure out if there's a way of convincing valgrind to dump 
info about all the currently allocated memory while the program is still 
running - there would be a lot of legitimate stuff in the report, but 
hopefully a few hundred MB of memory that shouldn't be there would stick 
out like a sore thumb.

> If you are using a 64-bit OS then the unaccounted numbers there are
> bogus. They are based on mallinfo() lookup results which suffer from
> 32-bit wrap issues. Use the OS command line to get a memory report
> from top or similar instead.

In this case I don't believe the data is bogus.  I have 8 workers, which 
top shows as:
26249 squid     20   0  871m 782m 5348 S  0.7 10.0   8:44.81 squid
26245 squid     20   0  732m 644m 5344 S  0.3  8.3   8:00.77 squid
26244 squid     20   0  706m 617m 5348 S  1.0  7.9   7:42.29 squid
26250 squid     20   0  699m 613m 5348 S  3.6  7.9   4:43.49 squid
26246 squid     20   0  699m 612m 5348 S  2.3  7.9   6:12.78 squid
26251 squid     20   0  662m 576m 5348 S  0.7  7.4   5:45.11 squid
26248 squid     20   0  649m 564m 5348 S  2.3  7.2   9:22.91 squid
26247 squid     20   0  603m 518m 5348 S  1.3  6.6   4:45.47 squid

Adding the sizes in the "virt" column gives me 5621MB.  The combined RSS 
is 4926MB.  Process 26250 has just 2MB in swap, the rest have no swap at 
all.  I guess the difference between the RSS and Virt totals can 
probably be almost entirely accounted for by the executables.

mallinfo() says there is about 4897MB in the arena - close enough to sum 
of the RSS that I'm inclined to believe it.  Since no one process 
exceeds 2GB, mallinfo() is probably trustworthy in this case.

The accounted for memory is 440MB - potentially higher than I would 
expect for a Squid with caching disabled, but certainly low enough to 
not be an immediate concern.  But that gives us about 4.4GB of memory 
unaccounted for (by subtracting 440MB from the sum of the RSS as shown 
by top).

4.4GB of unaccounted memory after running for just 6 hours is a 
significant amount, and if left to their own devices Squid continues to 
eat all the memory, leaving the machine thrashing swap.

I count 531 sockets in netstat, so my guess is that it isn't just data 
associated with some open sockets:
netstat -apn|egrep '26249|26245|26244|26250|26246|26251|26248|26247' -c
531


-- 
  - Steve