<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<blockquote type="cite">
<pre><blockquote type="cite">
Is there anything that you guys can suggest I do around the cache?
Should I try a different store type? A different filesystem type perhaps?
</blockquote>
If your store has a configuration knob that effectively limits disk
writing rate, then use it to limit that rate to avoid overflowing the queue.
You can also consider using rock store that has an explicit write rate
limiting option (but comes with other problems that may or may not
affect your setup).
Adding more physical disk spindles helps, of course.
</pre>
</blockquote>
<div>Thanks Alex. My next approach is to add more spindles. In fact I am currently testing with 4 cache dirs. All aufs currently. After adding these cache dirs, Squid started crashing every 20mins or so..</div>
<div><br>
</div>
<div>--cut-- (Cache.log)</div>
<div><span style="font-family:monospace"><span style="color:#000000;background-color:#ffffff;">FATAL: Received Segment Violation...dying.
</span><br>
2016/09/26 15:54:54 kid1| Closing HTTP port 0.0.0.0:8080 <br>
2016/09/26 15:54:54 kid1| storeDirWriteCleanLogs: Starting... <br>
2016/09/26 15:54:54 kid1| 65536 entries written so far. <br>
2016/09/26 15:54:54 kid1| Finished. Wrote 98159 entries. <br>
2016/09/26 15:54:54 kid1| Took 0.05 seconds (2044425.47 entries/sec). <br>
CPU Usage: 1230.157 seconds = 166.490 user + 1063.666 sys <br>
Maximum Resident Size: 3535840 KB <br>
Page faults with physical i/o: 0 <br>
2016/09/26 15:54:57 kid1| Set Current Directory to /var/cache/squid/ <br>
2016/09/26 15:54:57 kid1| Starting Squid Cache version 3.5.21 for x86_64-pc-linux-gnu...
<br>
2016/09/26 15:54:57 kid1| Service Name: squid <br>
2016/09/26 15:54:57 kid1| Process ID 16054 <br>
2016/09/26 15:54:57 kid1| Process Roles: worker <br>
2016/09/26 15:54:57 kid1| With 65536 file descriptors available <br>
2016/09/26 15:54:57 kid1| Initializing IP Cache... <br>
2016/09/26 15:54:57 kid1| DNS Socket created at 0.0.0.0, FD 7 <br>
2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf <br>
2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf <br>
2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf <br>
2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf <br>
2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf <br>
2016/09/26 15:54:57 kid1| Adding domain xxxxxx from /etc/resolv.conf <br>
2016/09/26 15:54:57 kid1| Adding domain </span><span style="font-family: monospace;">xxxxxx</span><span style="font-family:monospace"> from /etc/resolv.conf
<br>
2016/09/26 15:54:57 kid1| Adding domain </span><span style="font-family: monospace;">xxxxxx</span><span style="font-family:monospace"> from /etc/resolv.conf
<br>
2016/09/26 15:54:57 kid1| Adding nameserver </span><span style="font-family: monospace;">xxxxxx</span><span style="font-family:monospace"> from /etc/resolv.conf
<br>
2016/09/26 15:54:57 kid1| Adding nameserver </span><span style="font-family: monospace;">xxxxxx</span><span style="font-family:monospace"> from /etc/resolv.conf
<br>
2016/09/26 15:54:57 kid1| Adding nameserver </span><span style="font-family: monospace;">xxxxxx</span><span style="font-family:monospace"> from /etc/resolv.conf
<br>
2016/09/26 15:54:57 kid1| Logfile: opening log daemon:/var/log/squid//access.log <br>
2016/09/26 15:54:57 kid1| Logfile Daemon: opening log /var/log/squid//access.log <br>
2016/09/26 15:54:57 kid1| Store logging disabled <br>
2016/09/26 15:54:57 kid1| Swap maxSize 41943040 + 4194304 KB, estimated 3549026 objects
<br>
2016/09/26 15:54:57 kid1| Target number of buckets: 177451 <br>
2016/09/26 15:54:57 kid1| Using 262144 Store buckets <br>
2016/09/26 15:54:57 kid1| Max Mem size: 4194304 KB <br>
2016/09/26 15:54:57 kid1| Max Swap size: 41943040 KB <br>
2016/09/26 15:54:57 kid1| Rebuilding storage in /var/cache/squid-c1/ (clean log) <br>
2016/09/26 15:54:57 kid1| Rebuilding storage in /var/cache/squid-c2/ (clean log) <br>
2016/09/26 15:54:57 kid1| Rebuilding storage in /var/cache/squid-c3/ (clean log) <br>
2016/09/26 15:54:57 kid1| Rebuilding storage in /var/cache/squid-c4/ (clean log) <br>
2016/09/26 15:54:57 kid1| Using Least Load store dir selection <br>
2016/09/26 15:54:57 kid1| Set Current Directory to /var/cache/squid/ <br>
2016/09/26 15:54:57 kid1| Finished loading MIME types and icons. <br>
2016/09/26 15:54:57 kid1| HTCP Disabled. <br>
2016/09/26 15:54:57 kid1| Sending SNMP messages from 0.0.0.0:3401 <br>
2016/09/26 15:54:57 kid1| Configuring Parent </span><span style="font-family: monospace;">xxxxxx</span><span style="font-family:monospace">8/80/0
<br>
2016/09/26 15:54:57 kid1| Configuring Parent </span><span style="font-family: monospace;">xxxxxx</span><span style="font-family:monospace">/80/0
<br>
2016/09/26 15:54:57 kid1| Configuring Sibling </span><span style="font-family: monospace;">xxxxxx</span><span style="font-family:monospace">/8080/3131
<br>
2016/09/26 15:54:57 kid1| Squid plugin modules loaded: 0 <br>
2016/09/26 15:54:57 kid1| Adaptation support is off. <br>
2016/09/26 15:54:57 kid1| Accepting HTTP Socket connections at local=0.0.0.0:8080 remote=[::] FD 18 flags=9
<br>
2016/09/26 15:54:57 kid1| Accepting SNMP messages on 0.0.0.0:3401 <br>
2016/09/26 15:54:57 kid1| Store rebuilding is 33.00% complete <br>
2016/09/26 15:54:57 kid1| Done reading /var/cache/squid-c3/ swaplog (8660 entries)
<br>
2016/09/26 15:54:57 kid1| Done reading /var/cache/squid-c1/ swaplog (12119 entries)
<br>
2016/09/26 15:54:58 kid1| Done reading /var/cache/squid-c2/ swaplog (30684 entries)
<br>
2016/09/26 15:54:58 kid1| Done reading /var/cache/squid-c4/ swaplog (46696 entries)
<br>
2016/09/26 15:54:58 kid1| Finished rebuilding storage from disk. <br>
2016/09/26 15:54:58 kid1| 98159 Entries scanned <br>
2016/09/26 15:54:58 kid1| 0 Invalid entries. <br>
2016/09/26 15:54:58 kid1| 0 With invalid flags. <br>
2016/09/26 15:54:58 kid1| 98159 Objects loaded. <br>
2016/09/26 15:54:58 kid1| 0 Objects expired. <br>
2016/09/26 15:54:58 kid1| 0 Objects cancelled. <br>
2016/09/26 15:54:58 kid1| 0 Duplicate URLs purged. <br>
2016/09/26 15:54:58 kid1| 0 Swapfile clashes avoided. <br>
2016/09/26 15:54:58 kid1| Took 0.61 seconds (162197.49 objects/sec). <br>
2016/09/26 15:54:58 kid1| Beginning Validation Procedure <br>
2016/09/26 15:54:58 kid1| Completed Validation Procedure <br>
2016/09/26 15:54:58 kid1| Validated 98159 Entries <br>
2016/09/26 15:54:58 kid1| store_swap_size = 11495920.00 KB <br>
2016/09/26 15:54:58 kid1| storeLateRelease: released 0 objects<br>
</span></div>
<div><br>
</div>
<div>--cut--</div>
<div><br>
</div>
<div><br>
</div>
<div> I tried rock but was stuck with this error for a while and decided to go back to aufs until I can do some more reading up. </div>
<div><br>
</div>
<div><span style="font-family:monospace"><span style="color:#000000;background-color:#ffffff;">FATAL: Rock cache_dir at /var/cache/squid-c3//rock failed to open db file: (2) No such file or directory</span><br>
</span></div>
<div><br>
</div>
<div>I am currently testing again after recompiling squid.. The rest of my proxy cluster has the "cache deny all" configuration directive.. So nothing is cached.</div>
<div><br>
</div>
<br>
<br>
<mc type="body"><br>
<br>
<font face="Arial;" size="1">Disclaimer:<br>
<a href="http://www.shopriteholdings.co.za/Pages/ShopriteE-mailDisclaimer.aspx" target="_blank">http://www.shopriteholdings.co.za/Pages/ShopriteE-mailDisclaimer.aspx</a>
</font>
</body>
</html>