[squid-users] Very high CPU ussage after migration 2.7 -> 3.5

[Amos Jeffries]

On 18/04/2015 5:29 a.m., Sebastian Goicochea wrote:
> Something new:
> 
> Taking away all refresh patterns the CPU usage goes down to around 15%
> Patterns look like this, maybe I have to change something:
> 
> refresh_pattern -i ^http://c2r\.microsoft\.com/ConsumerC2R/(.*) 43200
> 100% 129600 override-expire ignore-reload  ignore-private override-lastmod
> 

Dropping that (.*) off the end would be useful. There is an implicit .*
on the start and end of all regex patterns unless you explicitly add teh
^ and $ anchors.

Removing it allows the regex library to stop trying to match this
pattern when the bit you actually care about has been identified.
Instead of continuing through the other 0-64KB of possible URL length
character...by...character.


> refresh_pattern -i
> ^http:\/\/.*\.avg\.com\/softw\/.*\/update\/(.*\.bin)   43200 100%
> 129600  override-expire ignore-reload  ignore-private override-lastmod
> 

What matters more is how many patterns you have, what library is being
used, and how long the traffic URLs are. Each pattern gets applied 1-2
times per request. Squids provided GNUregex is really very slow, almost
any other available is faster these days.


> 
> Thanks
> 
> El 17/04/15 a las 13:16, Sebastian Goicochea escribió:
>> Hello,
>>
>> I have migrated from Squid 2.7 to Squid 3.5 given the fact that I need
>> some of the new functions that newer versions have.
>> The problem I'm having is that squid proccess is using 100% of CPU all
>> the time. Cache.log informed squid was low on file descriptors, but
>> that's already solved.
>>
>> Now I see a lot of these in cache.log:
>>
>> 2015/04/17 13:01:07 kid1| Starting new store_id helpers...
>> 2015/04/17 13:01:07 kid1| helperOpenServers: Starting 1/30 'jesred'
>> processes
>> 2015/04/17 13:01:08 kid1| Starting new store_id helpers...
>> 2015/04/17 13:01:08 kid1| helperOpenServers: Starting 1/30 'jesred'
>> processes
>>
>> It always says 1/30, is it only using one rewriter? Most of the CPU
>> time is idle time, so maybe squid is waiting for something, but I
>> don't know what.

That means it is starting 1 *more*, out of a maximum of 30.

If you find these occuring often, increase the idle= helper parameter to
something lke 2,3,5 etc.

Amos