[squid-users] Is Squid can shutdown unused idle redirector's children?

Tue Feb 17 10:21:20 UTC 2015

17.02.15 6:07, Marcus Kool пишет:
>
>
> On 02/16/2015 08:00 PM, Eliezer Croitoru wrote:
>> Hey Yuri,
>>
>> OK I have seen something...
>> Now we might need also the virtual memory which might be vsz.
>> And the cachemgr output is not from squidview..
>> The last image I have seen from cachemgr was much helpful(with 10 
>> helpers).
>>
>>  From what I have seen until now squidGuard uses about 13 MB of ram 
>> constantly.
>> If this is what it costs to run a squidGuard helper you should 
>> consider that it has some peak usage time and lower bound usage time 
>> and you will need to calculate how much helpers you need to provide
>> smooth surfing on peak usage time.
>> You should consider this peak usage as the required memory from 
>> squidGuard for smooth operation.
>> Since not you and not will be able to change the basic memory 
>> footprint(required) for squidGuard you will need to somehow decide if 
>> you require a much efficient software or to adjust your system
>> accordingly.
>>
>> I would agree that 100 * 13 MB of ram means about 1.3 GB of ram usage 
>> and might not be acceptable for a "simple" filtering mechanism.
>
> It is because squidGuard uses Berkeley DB which uses a default cache 
> size of 256 KB for each URL table.
> And each squidGuard process does this.  And of course the OS keeps a 
> full copy in the file system buffer cache.
> So with 100 redirectors and N URL tables, N * 100 * 256K is used by 
> buffers for Berkeley DB alone.
>
>> The main reasons for the unknown need for 100 helpers might be since 
>> it was not designed to be used this way.
>
> squidGuard does not support the Squid feature 'concurrency' for 
> url_rewrite_children.  ufdbGuard does.
> With concurrency, latency goes down and the number of processes can 
> also be reduced.
The lack of concurrency is main disadvantage of squidGuard. Old 
process-based model is not acceptable on modern OS'es.

BTW, can I configure ufdbGuard children like this:

url_rewrite_children 1 startup=1 idle=1 concurrency=100

?

Another reason is store ID rewriter too. With this we have the same 
problem, but a bit smaller, of course.
I think, global parameter to firce shutdown idle redirectirs will be 
best solution.

>
>> I must say that squidGuard is a very good piece of software but it 
>> lacks couple things which might make it un-usable for some if not 
>> many systems and maybe including yours.
>> Indeed in your case there is a chance that if you will even install a 
>> full blown DB that will be stored all in ram and you will write a 
>> helper that will mimic squidguard functionality\logic with
>> concurrency support you will get much less memory consumption with 
>> much more efficient request handling.
>>
>> It is much smarter to write a helper which will have a overall 
>> efficiency mark on it then to add into squid something that might not 
>> be even needed in the first place.
>
> ufdbGuard has all that you need.  It holds one copy of the database in 
> memory without allocating extra memory.
> The ufdbGuard URL redirectors are leightweight processes using very 
> little memory and
> the redirectors support concurrency so less are required.
>
> Off list, Yuri asked help for compilation on Solaris and I made a fix 
> for ufdbGuard which is availabe for Yuri.
> If there are no further complaints from Yuri, I will release the patch 
> in one or two days.
Will try. Also will be good to support modern startup facility on 
Solaris ;) against deprecated SVR4 init. ;)
>
> Marcus
>
>> For now before writing any helper etc I will want to see the cachemgr 
>> interface output for the "redirector" option in two cases:
>> - startup
>> - after midnight when there are lots of helpers running under squid
>>
>> You can see that squid since version 3.2 squid offers access to the 
>> squid cache manager interface using a simple url such as:
>> "http://filter:3128/squid-internal-mgr/redirector"
>> (using the visible_hostname and the forward proxy port of squid)
>>
>> All The Bests,
>> Eliezer
>>
>> On 16/02/2015 21:56, Yuri Voinov wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Now:
>>>
>>> 2015/02/16 23:10:23 kid1|   store_swap_size = 29826351.50 KB
>>> 2015/02/16 23:10:24 kid1| storeLateRelease: released 0 objects
>>> 2015/02/16 23:15:01 kid1| Starting new redirector helpers...
>>> 2015/02/16 23:15:01 kid1| helperOpenServers: Starting 1/100
>>> 'squidGuard' processes
>>> 2015/02/17 01:40:15 kid1| Starting new redirector helpers...
>>> 2015/02/17 01:40:15 kid1| helperOpenServers: Starting 1/100
>>> 'squidGuard' processes
>>> ^Croot @ cthulhu / # ps -e -o user,pid,rss,comm
>>
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users