<div dir="ltr">I try to use jemalloc, but from monitoring, there is no difference, <div><br></div><div>I follow this guide:</div><div><a href="https://github.com/jemalloc/jemalloc/wiki/Getting-Started">https://github.com/jemalloc/jemalloc/wiki/Getting-Started</a><br></div><div><br></div><div>I used LD_PRELOAD to let squid use jemalloc.</div><div><br></div><div>is there something wrong ?</div><div><br></div><div>If I want to re-compile squid with jemalloc, how can i do that ? Can I just use some FLAGS ?</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">2015-11-28 0:23 GMT+08:00 Marcus Kool <span dir="ltr"><<a href="mailto:marcus.kool@urlfilterdb.com" target="_blank">marcus.kool@urlfilterdb.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I do not have the detail of Ubuntu 14.04 but most likely 12.04 and 14.04 have a different version of malloc (see "man malloc") which allocates gigabytes of virtual memory.<br>
Most likely you see in top that the resident memory is what you expect that Squid uses (comparable as on 12.04) and the virtual memory is high.<br>
This is the new "normal" behavior of malloc.<br>
<br>
Marcus<span class=""><br>
<br>
<br>
On 11/27/2015 01:41 PM, 风声 wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
Hi,<br>
<br>
We had installed squid 3.3.8 on ubuntu 12.04, when we upgrade the OS from ubuntu 12.04 to 14.04, we found the memory usage abnormal, eat up all memory.<br>
<br>
After we investigation:<br>
<br>
We use 4 servers for comparision:<br>
<br>
3.3.8 on ubuntu 12.04<br>
3.5.11 on ubuntu 12.04<br>
<br>
3.3.8 on ubuntu 14.04<br>
3.5.11 on ubuntu 14.04<br>
<br>
all squid on ubuntu 12.04 works normal, all squid on ubuntu 14.04 memory usage always keep going up, never goes down. But when I check the system info, can not find where the memory was used. If we<br>
restart the squid service, the memory comes back.<br>
<br>
<br>
Can you give me any hints ?<br>
<br>
Below is the info on abnormal server:<br>
<br></span>
*lsb_release -a*<span class=""><br>
No LSB modules are available.<br>
Distributor ID:Ubuntu<br>
Description:Ubuntu 14.04.3 LTS<br>
Release:14.04<br>
Codename:trusty<br>
<br></span>
*free -m*<span class=""><br>
total used free shared buffers cached<br>
Mem: 7480 6405 1074 0 222 668<br>
-/+ buffers/cache: 5513 1966<br>
Swap: 0 0 0<br>
<br></span>
*cat /proc/meminfo*<div><div class="h5"><br>
MemTotal: 7659544 kB<br>
MemFree: 1107944 kB<br>
Buffers: 228352 kB<br>
Cached: 685076 kB<br>
SwapCached: 0 kB<br>
Active: 1373880 kB<br>
Inactive: 199260 kB<br>
Active(anon): 663652 kB<br>
Inactive(anon): 316 kB<br>
Active(file): 710228 kB<br>
Inactive(file): 198944 kB<br>
Unevictable: 5300 kB<br>
Mlocked: 5300 kB<br>
SwapTotal: 0 kB<br>
SwapFree: 0 kB<br>
Dirty: 128 kB<br>
Writeback: 0 kB<br>
AnonPages: 665056 kB<br>
Mapped: 21012 kB<br>
Shmem: 712 kB<br>
Slab: 332904 kB<br>
SReclaimable: 77376 kB<br>
SUnreclaim: 255528 kB<br>
KernelStack: 1368 kB<br>
PageTables: 5068 kB<br>
NFS_Unstable: 0 kB<br>
Bounce: 0 kB<br>
WritebackTmp: 0 kB<br>
CommitLimit: 3829772 kB<br>
Committed_AS: 864212 kB<br>
VmallocTotal: 34359738367 kB<br>
VmallocUsed: 22316 kB<br>
VmallocChunk: 34359708548 kB<br>
HardwareCorrupted: 0 kB<br>
AnonHugePages: 587776 kB<br>
HugePages_Total: 0<br>
HugePages_Free: 0<br>
HugePages_Rsvd: 0<br>
HugePages_Surp: 0<br>
Hugepagesize: 2048 kB<br>
DirectMap4k: 51200 kB<br>
DirectMap2M: 7944192 kB<br>
<br></div></div>
*cat /proc/net/sockstat*<span class=""><br>
sockets: used 7086<br>
TCP: inuse 7173 orphan 354 tw 1995 alloc 7224 mem 455543<br>
UDP: inuse 9 mem 4<br>
UDPLITE: inuse 0<br>
RAW: inuse 1<br>
FRAG: inuse 0 memory 0<br>
<br>
<br></span>
*netstat -na | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'*<span class=""><br>
LAST_ACK 39<br>
LISTEN 5<br>
SYN_RECV 7<br>
CLOSE_WAIT 51<br>
ESTABLISHED 6004<br>
FIN_WAIT1 314<br>
FIN_WAIT2 54<br>
SYN_SENT 3<br>
TIME_WAIT 1687<br>
<br></span>
*slabtop -s c*<span class=""><br>
Active / Total Objects (% used) : 602429 / 841775 (71.6%)<br>
Active / Total Slabs (% used) : <a href="tel:28101%20%2F%2028101" value="+862810128101" target="_blank">28101 / 28101</a> (100.0%)<br>
Active / Total Caches (% used) : 64 / 97 (66.0%)<br>
Active / Total Size (% used) : 282169.82K / 328114.05K (86.0%)<br>
Minimum / Average / Maximum Object : 0.01K / 0.39K / 8.00K<br>
<br>
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME<br>
37136 37003 99% 4.00K 4642 8 148544K kmalloc-4096<br>
335283 173366 51% 0.10K 8597 39 34388K buffer_head<br>
104768 94358 90% 0.25K 3274 32 26192K kmalloc-256<br>
9776 7033 71% 2.00K 611 16 19552K kmalloc-2048<br>
9810 6588 67% 1.75K 545 18 17440K TCP<br>
13959 13959 100% 0.96K 423 33 13536K ext4_inode_cache<br>
55083 52199 94% 0.19K 2623 21 10492K dentry<br>
13856 8621 62% 0.50K 433 32 6928K kmalloc-512<br>
52064 49778 95% 0.12K 1627 32 6508K kmalloc-128<br>
9650 6515 67% 0.62K 386 25 6176K sock_inode_cache<br>
18798 8329 44% 0.30K 723 26 5784K nf_conntrack_ffffffff81cdab80<br>
27825 16442 59% 0.19K 1325 21 5300K kmalloc-192<br>
9240 5902 63% 0.55K 330 28 5280K radix_tree_node<br>
7952 7952 100% 0.57K 284 28 4544K inode_cache<br>
2880 1874 65% 1.00K 90 32 2880K kmalloc-1024<br>
16992 16992 100% 0.11K 472 36 1888K sysfs_dir_cache<br>
2875 2875 100% 0.63K 115 25 1840K proc_inode_cache<br>
<br>
<br></span>
*squid3 -v*<span class=""><br>
Squid Cache: Version 3.5.11<br>
Service Name: squid<br>
configure options: '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc'<br>
'--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.' '--disable-maintainer-mode' '--disable-dependency-tracking' '--disable-silent-rules' '--datadir=/usr/share/squid3'<br>
'--sysconfdir=/etc/squid3' '--mandir=/usr/share/man' '--enable-inline' '--with-aufs-threads=8' '--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap' '--enable-delay-pools'<br>
'--enable-cache-digests' '--enable-underscores' '--enable-follow-x-forwarded-for' '--enable-auth-basic=NCSA' '--enable-auth-digest=file' '--enable-htcp' '--enable-url-rewrite-helpers=fake'<br>
'--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos' '--disable-auth-negotiate' '--disable-auth-ntlm' '--disable-ecap' '--disable-external-acl-helpers' '--disable-icap-client'<br>
'--disable-ipv6' '--disable-translation' '--with-swapdir=/var/spool/squid3' '--with-logdir=/var/log/squid3' '--with-pidfile=/var/run/squid3.pid' '--with-filedescriptors=100000' '--with-large-files'<br>
'--with-default-user=proxy' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall'<br>
'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security'<br>
<br>
<br></span>
_______________________________________________<br>
squid-users mailing list<br>
<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
<br>
</blockquote>
_______________________________________________<br>
squid-users mailing list<br>
<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
</blockquote></div><br></div>