[squid-users] Crash: every 1-2 hour: kernel: Out of memory: Kill process (squid)

Wed Dec 14 11:40:10 UTC 2016

Hello. I wrote earlier in wrong location:
http://bugs.squid-cache.org/show_bug.cgi?id=4647

> Squid eats all RAM, then eats all swap in a hour and killed by kernel.
>I was try to turn off cache, change squid version, change some
configuration parameters by this guide
http://wiki.squid-cache.org/SquidFaq/SquidMemory except malloc, but nothing
helps.

I made some config changes in accordance with the advice of Amos Jeffries
(via on). But it does not help.
This trouble somehow linked with https.
If wccp redirects only 80 port - works fine.
  wccp2_service_info 70 protocol = tcp flags = dst_ip_hash priority = 231
ports = 80
If wccp redirects 443 too - then squid overflows and killed by kernel
  wccp2_service_info 70 protocol = tcp flags = dst_ip_hash priority = 231
ports = 80,443

---Before it died (HTTPS on):
Mem:  16291720k total, 16125288k used,   166432k free,      540k buffers
Swap:  8216568k total,  8112628k used,   103940k free,    27112k cached
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
30858 squid     20   0 22.7g  14g 3612 S  8.0 94.6  14:50.82 squid

# free -m
             total       used       free     shared    buffers     cached
Mem:         15909      15750        158          0          0         26
-/+ buffers/cache:      15723        186
Swap:         8023       7936         87

Start Time:	Sat, 10 Dec 2016 07:52:50 GMT
Current Time:	Sat, 10 Dec 2016 09:39:45 GMT

Connection information for squid:
	Number of clients accessing cache:	1305
	Number of HTTP requests received:	193434
	Number of ICP messages received:	0
	Number of ICP messages sent:	0
	Number of queued ICP replies:	0
	Number of HTCP messages received:	0
	Number of HTCP messages sent:	0
	Request failure ratio:	 0.00
	Average HTTP requests per minute since start:	1809.2
	Average ICP messages per minute since start:	0.0
	Select loop called: 4529796 times, 1.416 ms avg
Cache information for squid:
	Hits as % of all requests:	5min: 0.0%, 60min: 0.0%
	Hits as % of bytes sent:	5min: 0.1%, 60min: -0.0%
	Memory hits as % of hit requests:	5min: 0.0%, 60min: 0.0%
	Disk hits as % of hit requests:	5min: 0.0%, 60min: 0.0%
	Storage Swap size:	82044 KB
	Storage Swap capacity:	80.1% used, 19.9% free
	Storage Mem size:	107876 KB
	Storage Mem capacity:	20.6% used, 79.4% free
	Mean Object Size:	29.54 KB
	Requests given to unlinkd:	9258
Median Service Times (seconds)  5 min    60 min:
	HTTP Requests (All):   0.10857  0.04519
	Cache Misses:          0.01648  0.00678
	Cache Hits:            0.00000  0.00000
	Near Hits:             0.00000  0.00000
	Not-Modified Replies:  0.00000  0.00000
	DNS Lookups:           0.00860  0.00779
	ICP Queries:           0.00000  0.00000
Resource usage for squid:
	UP Time:	6415.101 seconds
	CPU Time:	902.767 seconds
	CPU Usage:	14.07%
	CPU Usage, 5 minute avg:	15.97%
	CPU Usage, 60 minute avg:	13.96%
	Maximum Resident Size: 62241760 KB
	Page faults with physical i/o: 32647
Memory accounted for:
	Total accounted:       1073388 KB
	memPoolAlloc calls:     12969
	memPoolFree calls:   35802441
File descriptor usage for squid:
	Maximum number of file descriptors:   100000
	Largest file desc currently in use:   28744
	Number of file desc currently in use: 28738
	Files queued for open:                   0
	Available number of file descriptors: 71262
	Reserved number of file descriptors:   100
	Store Disk files open:                   0
Internal Data Structures:
	 57337 StoreEntries
	 54560 StoreEntries with MemObjects
	    52 Hot Object Cache Items
	  2777 on-disk objects

---after:
/var/log/messages
kernel: 11733 total pagecache pages
kernel: 8957 pages in swap cache
kernel: Swap cache stats: add 21118384, delete 21109427, find
12110273/12422740
kernel: Free swap  = 0kB
kernel: Total swap = 8216568kB
kernel: 4194303 pages RAM
kernel: 121373 pages reserved
kernel: 11781 pages shared
kernel: 4023631 pages non-shared
...omitted...
kernel: Out of memory: Kill process 30858 (squid) score 954 or sacrifice
child
kernel: Killed process 30868, UID 23, (log_file_daemon) total-vm:26640kB,
anon-rss:48kB, file-rss:512kB
(squid-1): I don't handle this error well!
Dec 10 12:44:27 localhost squid[30855]: Squid Parent: (squid-1) process
30858 exited due to signal 9 with status 0

In attach all /var/log/messages output.
Main task for the server is to block bad sites and bypass others on same
IPs.
Any ideas?

--
Sergey

-------------- next part --------------
A non-text attachment was scrubbed...
Name: squid.conf
Type: application/octet-stream
Size: 2470 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20161214/f39b395f/attachment-0002.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: mgr-info_only-http.txt
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20161214/f39b395f/attachment-0001.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: messages_when_died
Type: application/octet-stream
Size: 11847 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20161214/f39b395f/attachment-0003.obj>