<div dir="ltr">Hi Alex,<div><br></div><div>BTW, where do I send the files with the memory logs? I had sent a few attachments in an email reply to this thread but got back a message saying it is too large and will need to be reviewed by an admin.</div><div>Thanks</div><div>Praveen</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Dec 22, 2021 at 10:58 PM Praveen Ponakanti <<a href="mailto:pponakanti@roblox.com">pponakanti@roblox.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi Alex,<div><br></div><div>Thanks, please see inline below.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Dec 22, 2021 at 6:45 AM Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 12/21/21 7:48 PM, Praveen Ponakanti wrote:<br>
<br>
> We are running the squid proxy for servicing outbound HTTP quests from<br>
> our network and have observed a significant memory leak with 5.x<br>
> versions. While there are several discussions about memory leaks with<br>
> recent versions, just wanted to list out what we have observed in case<br>
> this is an unknown leak.<br>
<br>
I recommend sharing a log with 48+ hourly mgr:mem snapshots. These<br>
snapshots help compare your leak with others we know about and may help<br>
isolate some of the memory leaks:<br>
<a href="https://bugs.squid-cache.org/show_bug.cgi?id=5132#c8" rel="noreferrer" target="_blank">https://bugs.squid-cache.org/show_bug.cgi?id=5132#c8</a><br><br></blockquote><div><br></div><div>Attached the current memory stats from the 2 nodes that have been taking a similar rate of requests, with the node running 5.3 already using up 20G more than the 4.17 version. The memory leak increases over a few hours in almost a step-like curve each day on the 5.x versions; I will continue collecting these stats hourly.</div><div><br></div><div>Note, I am using a curl to <a href="http://localhost:3128/squid-internal-mgr/mem" target="_blank">http://localhost:3128/squid-internal-mgr/mem</a> to collect the memory stats as we dont have the squidclient built into the docker container that runs the squid proxy. The output appears to be quite large with data for each kid process (we have 30 workers configured).</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> I have attempted to build squid with -with-valgrind-debug, and run it in<br>
> a test env. However valgrind appears to collect some data from the parse<br>
> config functions and then the squid proxy restarts. <br>
<br>
FWIW, valgrind works as expected in my environment. If your Squid proxy<br>
is killed by an assertion or crashes, even in a test environment, please<br>
consider reporting that bug (after checking that it has not been<br>
reported already, of course). If your test proxy is misconfigured, then<br>
please fix the configuration before proceeding with tests.<br>
<br></blockquote><div>The Valgrind image isnt crashing, and it uses the same config that works in prod without valgrind. I might have to fix how it is launched from the docker entrypoint script in my test env. </div><div><br></div><div>root 117051 117005 0 Dec11 ? 00:00:00 /usr/bin/valgrind.bin --leak-check=full --show-leak-kinds=all --verbose --log-file=/squid/var/logs/valgrind-out.txt /squid/sbin/squid -f /squid/etc/squid.conf</div><div> </div><div>gdb attach 1<br>GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2<br><br>(gdb) monitor<br>"monitor" command not supported by this target.<br>(gdb)<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Thank you,<br>
<br>
Alex.<br>
_______________________________________________<br>
squid-users mailing list<br>
<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
</blockquote></div></div>
</blockquote></div>