<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
skyrocketing = using our maximum link download bandwidth.<br>
This machine is only proxying. Not being a firewall, not a router,
nor a gateway. It has access to the internet through our
gateway/firewall (pfsense).<br>
Lots of LAN clients are connected to the proxy, this is their only
way to the internet. 1 interface, debian linux. EXT4 FS. CPU/MEM
usage is always stable.<br>
Clients use it explicitly or via wpad. Never transparently. Now I'm
using 3 workers, because 1 is not enough and we have spare cores.<br>
It's a VM machine with netapp storage. lots of raid disks. <br>
SQUID was running perfectly without cache_dirs. <br>
<br>
I think squid is downloading and redownloading the same files over
and over again because: 1- these are segmented downloads and
range_offset_limit is set to NONE for these files. 2- it can't store
the downloaded files on the cache but I don't know why!<br>
<br>
1457358960.737 8399 10.23.0.63 <b>TCP_SWAPFAIL_MISS</b>/206
1058138 GET
<a class="moz-txt-link-freetext" href="http://au.download.windowsupdate.com/c/msdownload/update/software/uprl/2013/10/ie11-windows6.1-x64-en-us_ddec9ddc256ffa7d97831af148f6cc45130c6857.exe">http://au.download.windowsupdate.com/c/msdownload/update/software/uprl/2013/10/ie11-windows6.1-x64-en-us_ddec9ddc256ffa7d97831af148f6cc45130c6857.exe</a>
- HIER_DIRECT/201.30.251.43 application/octet-stream<br>
1457358987.869 22464 10.88.10.5 <b>TCP_SWAPFAIL_MISS</b>/206
1416417 GET
<a class="moz-txt-link-freetext" href="http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3141092-x64_0f7a98b9dc9f5c7ac73f6f543bf004a15e4d7be8.psf">http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3141092-x64_0f7a98b9dc9f5c7ac73f6f543bf004a15e4d7be8.psf</a>
- HIER_DIRECT/201.30.251.26 application/octet-stream<br>
<br>
Squid Object Cache: Version 3.5.15-20160301-r13999<br>
Build Info: <br>
Service Name: squid<br>
Start Time: Fri, 04 Mar 2016 21:15:47 GMT<br>
Current Time: Mon, 07 Mar 2016 18:14:37 GMT<br>
Connection information for squid:<br>
Number of clients accessing cache: 4543<br>
Number of HTTP requests received: 7305505<br>
Number of ICP messages received: 0<br>
Number of ICP messages sent: 0<br>
Number of queued ICP replies: 0<br>
Number of HTCP messages received: 0<br>
Number of HTCP messages sent: 0<br>
Request failure ratio: 0.00<br>
Average HTTP requests per minute since start: 1765.1<br>
Average ICP messages per minute since start: 0.0<br>
Select loop called: 365044476 times, 11.146 ms avg<br>
Cache information for squid:<br>
Hits as % of all requests: 5min: 22.0%, 60min: 21.2%<br>
Hits as % of bytes sent: 5min: 3.9%, 60min: 7.3%<br>
Memory hits as % of hit requests: 5min: 2.5%, 60min:
2.9%<br>
Disk hits as % of hit requests: 5min: 28.1%, 60min: 26.8%<br>
Storage Swap size: 179951936 KB<br>
Storage Swap capacity: 45.1% used, 54.9% free<br>
Storage Mem size: 4194176 KB<br>
Storage Mem capacity: 100.0% used, 0.0% free<br>
Mean Object Size: 46.07 KB<br>
Requests given to unlinkd: 0<br>
Median Service Times (seconds) 5 min 60 min:<br>
HTTP Requests (All): 0.06514 0.07007<br>
Cache Misses: 0.09730 0.10075<br>
Cache Hits: 0.00055 0.00055<br>
Near Hits: 0.06521 0.06757<br>
Not-Modified Replies: 0.00055 0.00055<br>
DNS Lookups: 0.00019 0.00019<br>
ICP Queries: 0.00000 0.00000<br>
Resource usage for squid:<br>
UP Time: 248329.183 seconds<br>
CPU Time: 21525.705 seconds<br>
CPU Usage: 8.67%<br>
CPU Usage, 5 minute avg: 52.35%<br>
CPU Usage, 60 minute avg: 47.90%<br>
Maximum Resident Size: 85809792 KB<br>
Page faults with physical i/o: 104<br>
Memory accounted for:<br>
Total accounted: 164356 KB<br>
memPoolAlloc calls: 1801863396<br>
memPoolFree calls: 1811836496<br>
File descriptor usage for squid:<br>
Maximum number of file descriptors: 81920<br>
Largest file desc currently in use: 6157<br>
Number of file desc currently in use: 8216<br>
Files queued for open: 0<br>
Available number of file descriptors: 73704<br>
Reserved number of file descriptors: 500<br>
Store Disk files open: 11<br>
Internal Data Structures:<br>
25054 StoreEntries<br>
391 StoreEntries with MemObjects<br>
96086 Hot Object Cache Items<br>
3905959 on-disk objects<br>
<br>
Best Regards,<br>
<br>
<pre class="moz-signature" cols="72">--
Heiler Bemerguy - (91) 98151-4894
Assessor Técnico - CINBESA (91) 3184-1751</pre>
<br>
<div class="moz-cite-prefix">Em 07/03/2016 14:33, Eliezer Croitoru
escreveu:<br>
</div>
<blockquote cite="mid:56DDBB65.3090907@ngtech.co.il" type="cite">On
07/03/2016 16:29, Heiler Bemerguy wrote:
<br>
<blockquote type="cite">We're still getting all these SWAPFAIL and
our link is
<br>
skyrocketing...... please help! I think it didn't happen on
older
<br>
versions (.14 and below)
<br>
</blockquote>
<br>
Hey,
<br>
<br>
What do you mean by skyrocketing?? like in the graph??
<br>
Also it is not clear to me something about the machine, is this
machine a FW\ROUTER\GW?
<br>
If so is it for a lan?
<br>
How many interface this machine has?
<br>
Is it a pfsense? if so what version?
<br>
What is the FS used for the cache directories?
<br>
Did you also measured CPU when you see the spikes? if so what is
it?
<br>
How clients access the proxy service? transparently or using a
browser setings or WPAD with dhcp settings?
<br>
Also I have seen you are using 2 workers, is it because one worker
doesn't seem to do the job?
<br>
Did you tried to change the values of:
<br>
cache_swap_low 98
<br>
cache_swap_high 99
<br>
<br>
from this high to lower numbers such as:
<br>
cache_swap_low 90
<br>
cache_swap_high 95
<br>
<br>
or even lower?
<br>
cache_swap_low 80
<br>
cache_swap_high 85
<br>
<br>
I am unsure about this since in the docs at:
<br>
<a class="moz-txt-link-freetext" href="http://www.squid-cache.org/Doc/config/cache_swap_low/">http://www.squid-cache.org/Doc/config/cache_swap_low/</a>
<br>
<a class="moz-txt-link-freetext" href="http://www.squid-cache.org/Doc/config/cache_swap_high/">http://www.squid-cache.org/Doc/config/cache_swap_high/</a>
<br>
<br>
the ROCK storage is not mentioned.
<br>
<br>
Also on what hardware are you running? what disks?
<br>
<br>
All the above are important and in your case it is possible that
there is something wrong in how the network is planned or the
software doing something wrong.
<br>
<br>
In scenarios like this I offer to verify two things:
<br>
- test what happens when you disable disk cache.(from CPU,
bandwidth, DISK aspect)
<br>
- dump the cache manager info page to see basic statistics about
the proxy traffic using:
<a class="moz-txt-link-freetext" href="http:/cache_ip_or_visiblie_host_name:3128/squid-internal-mgr/info">http:/cache_ip_or_visiblie_host_name:3128/squid-internal-mgr/info</a>
<br>
<br>
For scenarios like this I started working on a logging\monitoring
service\script that will run in the background of the machine and
will dump content of some statistics to enable couple squid
developers eyes to see these and then have a better understanding
the nature of the issue.
<br>
For now the script\service is not ready and will not be able to
help us so we need these dumps and information..
<br>
<br>
Eliezer
<br>
_______________________________________________
<br>
squid-users mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:squid-users@lists.squid-cache.org">squid-users@lists.squid-cache.org</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.squid-cache.org/listinfo/squid-users">http://lists.squid-cache.org/listinfo/squid-users</a>
<br>
</blockquote>
<br>
</body>
</html>