[squid-users] Optimizing squid

Thu Feb 25 16:58:50 UTC 2016

Hi Alex, Eliezer, Yuri, Amos..

So, to start from the start, after seeing squid was totally stable and 
fast, running with NO cache_dirs, I tried to add only 2 rockstore 
cache_dirs to test.

conf:
/cache_dir rock /cache2/rock1 20000 min-size=0 max-size=4096 
slot-size=2048//
//cache_dir rock /cache2/rock2 30000 min-size=4097 max-size=16384 
slot-size=4096/
(ps.: I know it would be nice to use one store PER partition/disk/lun 
whatever.. but I'm trying to lessen disk wasting by using small 
slot-sizes for small files.. am I wrong?)

Then squid -z:
/2016/02/25 13:42:00 kid2| Creating Rock db: /cache2/rock1/rock//
//2016/02/25 13:42:00 kid3| Creating Rock db: /cache2/rock2/rock/

Then running squid for the first time with these newly created rock 
stores....

/2016/02/25 13:42:09 kid3| Loading cache_dir #1 from /cache2/rock2/rock//
//2016/02/25 13:42:09 kid2| Loading cache_dir #0 from /cache2/rock1/rock//
//2016/02/25 13:42:09 kid3| Store rebuilding is 0.01% complete//
//2016/02/25 13:42:09 kid2| Store rebuilding is 0.01% complete/

Rebuilding what? just creating the huge files I think...

Then:
/2016/02/25 13:42:19 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:21 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 6943832064
2016/02/25 13:42:22 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 7424217088
2016/02/25 13:42:22 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 7616405504
2016/02/25 13:42:22 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 7813527552
2016/02/25 13:42:22 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 8124858368
2016/02/25 13:42:24 kid3| Store rebuilding is 42.67% complete
2016/02/25 13:42:24 kid2| Store rebuilding is 42.57% complete
2016/02/25 13:42:24 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:24 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:25 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:26 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:27 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:27 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:27 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:28 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 11881381888
2016/02/25 13:42:29 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 12680894464
2016/02/25 13:42:29 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 12737179648
2016/02/25 13:42:31 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:31 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 14243010560
2016/02/25 13:42:32 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 14443577344
2016/02/25 13:42:34 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 15603302400
2016/02/25 13:42:35 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 16192946176
2016/02/25 13:42:37 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:37 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:38 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:39 kid3| Store rebuilding is 86.02% complete
2016/02/25 13:42:39 kid2| Store rebuilding is 88.28% complete
2016/02/25 13:42:39 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:39 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:40 kid1| ctx: enter level  0: 
'http://static.bn-static.com/pg/0plcB0QjJpBbwN7rMMDjKKO5Z63Nhu3zfPw==.gif'
2016/02/25 13:42:40 kid1| WARNING: swapfile header inconsistent with 
available data
2016/02/25 13:42:40 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 19581075456
2016/02/25 13:42:41 kid2| WARNING: cache_dir[0]: Ignoring malformed 
cache entry meta data at 19757760512
2016/02/25 13:42:43 kid2| Finished rebuilding storage from disk.
2016/02/25 13:42:43 kid2|   10239992 Entries scanned
2016/02/25 13:42:43 kid2|        14 Invalid entries.///

What entry? why malformed? Wasn't it just a empty store?! it just 
created it.......

Best Regards,

-- 
Heiler Bemerguy - (91) 98151-4894
Assessor Técnico - CINBESA (91) 3184-1751

Em 24/02/2016 19:00, Alex Rousskov escreveu:
> On 02/24/2016 12:44 PM, Heiler Bemerguy wrote:
>
>> I don't think I had a bottleneck on I/O itself,
> In general, I/O bottlenecks with rock or ufs cache_dirs do not result in
> sustained 100% CPU utilization. I do not know enough about aufs to say
> whether that cache_dir type can have those symptoms during overload.
>
>
>> maybe the hash/search of cache indexes was too much for a single thread?
> See my earlier response regarding Squid bugs when handling huge files.
> Nothing you have posted so far disproves that working theory.
>
>
> Cheers,
>
> Alex.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160225/32e7de92/attachment.html>