[squid-users] squid cache takes a break

[Yuri]

It is just enough not to reinvent the wheel. What needs op - already
exists and is called ufdbguard. And it's works perfectly with shallalist :)


13.09.2017 2:51, Eliezer Croitoru пишет:
> I just must add that if you understand how TCP works(which the helpers use to communicate with squid) then it makes sense that it is possible that...
> The sender (ie squid) sent 100 lines but the client software yet to process them since it's in the OS or other software\hardware related buffer.
>
> For me it was hard to understand at first since it's an STDIN\STDOUT interface so it would block after every write but it's not...
>
> There is a possibility that if the helper can process every incoming request with threading or other method of concurrency then the performance of the helper and by that squid will be better but if only using the basic buffer works fine for you then great.
>
> Eliezer
>
> ----
> Eliezer Croitoru
> Linux System Administrator
> Mobile: +972-5-28704261
> Email: eliezer at ngtech.co.il
>
>
>
> -----Original Message-----
> From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Amos Jeffries
> Sent: Tuesday, September 12, 2017 16:08
> To: squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] squid cache takes a break
>
> On 12/09/17 22:59, Vieri wrote:
>> ________________________________
>> From: Amos Jeffries <squid3 at treenet.co.nz>
>>> That is all it needs to do to begin with; parse off the numeric value
>>> from the input line and send it back as prefix on the output line. The
>>> helper does not need threading or anything particularly special for the > minimal support.
>>
>> I thought it had to be asynchronous.
>> The docs say "Only used with helpers capable of processing more than one query at a time."
>>
>> Example:
>> Squid sends "1 URI1" (or whatever) to the helper.
>> It does not wait for an immediate response.
>> In fact, Squid can send "2 URI2" before getting the reply to ID 1, right?
> Yes.
>
>
>> In my case, the helper is synchronous, non-MT. I don't think it will improve the time responses per-se.
>>
>> In any case, my helper won't be able to process more than one query AT A TIME.
>>
>> I tried it anyway. So here's the relevant code:
>>
>> while( <STDIN> )
>> {
>> s/^\s*//;
>> s/\s*$//;
>> my @squidin = split;
>> my $squidn = scalar @squidin;
>> undef $url;
>> undef $channelid;
>> if ( ($squidn == 2 ) && (defined $squidin[0]) && ($squidin[0] =~ /^\d+?$/) ) {
>> $channelid = $squidin[0];
>> $url = $squidin[1] if (defined $squidin[1]);
>> } else {
>> $url = $squidin[0] if (defined $squidin[0]);
>> }
>>
>> [...]
>> logtofile("Channel-ID: ".$channelid."\n") if ((defined $channelid) && ($debug >= 1));
>>
>> [...do DB lookups, reply accordingly...]
>>
>> if (defined $channelid) {
>> print( $channelid." OK\n" );
>> logtofile( $channelid." OK\n" ) if ($debug >= 1);
>> } else {
>> print( "OK\n" );
>> logtofile( "OK\n" ) if ($debug >= 1);
>> }
>> [...similar responses for ERR messages...]
>> }
>>
>> Here's the relevant squid.conf line:
>> external_acl_type bllookup ttl=86400 negative_ttl=86400 children-max=80 children-startup=10 children-idle=3 concurrency=8 %URI /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl [...]
>>
>> How can I check in the Squid log that concurrency is "working"?
> Section 82, level 2 or 4 should log the queries.
>
> Better info is in the cachemgr/squidclient "external_acl" report. Each 
> helper is listed with its total and summary stats for each helper child.
>
>> If the helper logs to a text file as in the trimmed code above, I notice that the channel ID is always 0. I get messages such as:
>> Channel-ID: 0
>> 0 OK
>> Channel-ID: 0
>> 0 ERR ...
>>
>> Is this expected?
> Maybe.
>
> If you make the helper pause a bit and throw a large number of different 
> URLs at Squid you should see it grow a bit higher than 0.
>
>> Despite this, I can see that the number of helper processes does not increase over time for now, and that HTTP/S client browsing is responsive enough.
>> # ps aux | grep -c squid_url_lookup.pl
>> 11
>>
> Yay.
>
>> One last thing. I'm using:
>> cache_dir diskd /var/cache/squid 100 16 256
>> I may want to try to comment out this directive for improved I/O performance.
>>
>> Thanks,
>>
>> Vieri
> Cheers
> Amos
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20170913/dc10aca7/attachment-0001.sig>