[squid-users] Crash: every 1-2 hour: kernel: Out of memory: Kill process (squid)

Antony Stone Antony.Stone at squid.open.source.it
Mon Dec 19 11:39:50 UTC 2016


On Monday 19 December 2016 at 12:34:19, noc at forceline.net wrote:

> Anybody can help?
> Maby I need to change mailing list?

Did you try the suggestions already made?

http://lists.squid-cache.org/pipermail/squid-users/2016-December/013777.html
http://lists.squid-cache.org/pipermail/squid-users/2016-December/013778.html
http://lists.squid-cache.org/pipermail/squid-users/2016-December/013780.html

Antony.

> > -----Original Message-----
> > From: noc at forceline.net [mailto:noc at forceline.net]
> > Sent: Wednesday, December 14, 2016 2:40 PM
> > To: 'squid-users at lists.squid-cache.org'
> > Subject: Crash: every 1-2 hour: kernel: Out of memory: Kill process
> > (squid)
> > 
> > 
> > Hello. I wrote earlier in wrong location:
> > http://bugs.squid-cache.org/show_bug.cgi?id=4647
> > 
> > > Squid eats all RAM, then eats all swap in a hour and killed by
> > 
> > kernel.
> > 
> > >I was try to turn off cache, change squid version, change some
> > 
> > configuration parameters by this guide http://wiki.squid-
> > cache.org/SquidFaq/SquidMemory except malloc, but nothing helps.
> > 
> > I made some config changes in accordance with the advice of Amos
> > Jeffries (via on). But it does not help.
> > This trouble somehow linked with https.
> > If wccp redirects only 80 port - works fine.
> > 
> >   wccp2_service_info 70 protocol = tcp flags = dst_ip_hash priority =
> > 
> > 231 ports = 80
> > If wccp redirects 443 too - then squid overflows and killed by kernel
> > 
> >   wccp2_service_info 70 protocol = tcp flags = dst_ip_hash priority =
> > 
> > 231 ports = 80,443
> > 
> > ---Before it died (HTTPS on):
> > Mem:  16291720k total, 16125288k used,   166432k free,      540k
> > buffers
> > Swap:  8216568k total,  8112628k used,   103940k free,    27112k cached
> > 
> >   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> > 
> > 30858 squid     20   0 22.7g  14g 3612 S  8.0 94.6  14:50.82 squid
> > 
> > 
> > # free -m
> > 
> >              total       used       free     shared    buffers
> > 
> > cached
> > Mem:         15909      15750        158          0          0
> > 26
> > -/+ buffers/cache:      15723        186
> > Swap:         8023       7936         87
> > 
> > 
> > Start Time:	Sat, 10 Dec 2016 07:52:50 GMT
> > Current Time:	Sat, 10 Dec 2016 09:39:45 GMT
> > 
> > Connection information for squid:
> > 	Number of clients accessing cache:	1305
> > 	Number of HTTP requests received:	193434
> > 	Number of ICP messages received:	0
> > 	Number of ICP messages sent:	0
> > 	Number of queued ICP replies:	0
> > 	Number of HTCP messages received:	0
> > 	Number of HTCP messages sent:	0
> > 	Request failure ratio:	 0.00
> > 	Average HTTP requests per minute since start:	1809.2
> > 	Average ICP messages per minute since start:	0.0
> > 	Select loop called: 4529796 times, 1.416 ms avg
> > 
> > Cache information for squid:
> > 	Hits as % of all requests:	5min: 0.0%, 60min: 0.0%
> > 	Hits as % of bytes sent:	5min: 0.1%, 60min: -0.0%
> > 	Memory hits as % of hit requests:	5min: 0.0%, 60min: 0.0%
> > 	Disk hits as % of hit requests:	5min: 0.0%, 60min: 0.0%
> > 	Storage Swap size:	82044 KB
> > 	Storage Swap capacity:	80.1% used, 19.9% free
> > 	Storage Mem size:	107876 KB
> > 	Storage Mem capacity:	20.6% used, 79.4% free
> > 	Mean Object Size:	29.54 KB
> > 	Requests given to unlinkd:	9258
> > 
> > Median Service Times (seconds)  5 min    60 min:
> > 	HTTP Requests (All):   0.10857  0.04519
> > 	Cache Misses:          0.01648  0.00678
> > 	Cache Hits:            0.00000  0.00000
> > 	Near Hits:             0.00000  0.00000
> > 	Not-Modified Replies:  0.00000  0.00000
> > 	DNS Lookups:           0.00860  0.00779
> > 	ICP Queries:           0.00000  0.00000
> > 
> > Resource usage for squid:
> > 	UP Time:	6415.101 seconds
> > 	CPU Time:	902.767 seconds
> > 	CPU Usage:	14.07%
> > 	CPU Usage, 5 minute avg:	15.97%
> > 	CPU Usage, 60 minute avg:	13.96%
> > 	Maximum Resident Size: 62241760 KB
> > 	Page faults with physical i/o: 32647
> > 
> > Memory accounted for:
> > 	Total accounted:       1073388 KB
> > 	memPoolAlloc calls:     12969
> > 	memPoolFree calls:   35802441
> > 
> > File descriptor usage for squid:
> > 	Maximum number of file descriptors:   100000
> > 	Largest file desc currently in use:   28744
> > 	Number of file desc currently in use: 28738
> > 	Files queued for open:                   0
> > 	Available number of file descriptors: 71262
> > 	Reserved number of file descriptors:   100
> > 	Store Disk files open:                   0
> > 
> > Internal Data Structures:
> > 	 57337 StoreEntries
> > 	 54560 StoreEntries with MemObjects
> > 	 
> > 	    52 Hot Object Cache Items
> > 	  
> > 	  2777 on-disk objects
> > 
> > ---after:
> > /var/log/messages
> > kernel: 11733 total pagecache pages
> > kernel: 8957 pages in swap cache
> > kernel: Swap cache stats: add 21118384, delete 21109427, find
> > 12110273/12422740
> > kernel: Free swap  = 0kB
> > kernel: Total swap = 8216568kB
> > kernel: 4194303 pages RAM
> > kernel: 121373 pages reserved
> > kernel: 11781 pages shared
> > kernel: 4023631 pages non-shared
> > ...omitted...
> > kernel: Out of memory: Kill process 30858 (squid) score 954 or
> > sacrifice child
> > kernel: Killed process 30868, UID 23, (log_file_daemon) total-
> > vm:26640kB, anon-rss:48kB, file-rss:512kB
> > (squid-1): I don't handle this error well!
> > Dec 10 12:44:27 localhost squid[30855]: Squid Parent: (squid-1) process
> > 30858 exited due to signal 9 with status 0
> > 
> > 
> > In attach all /var/log/messages output.
> > Main task for the server is to block bad sites and bypass others on
> > same IPs.
> > Any ideas?
> > 
> > --
> > Sergey

-- 
What makes you think I know what I'm talking about?
I just have more O'Reilly books than most people.

                                                   Please reply to the list;
                                                         please *don't* CC me.


More information about the squid-users mailing list