[squid-users] squid cache takes a break

[Amos Jeffries]

On 12/09/17 22:59, Vieri wrote:
> 
> ________________________________
> From: Amos Jeffries <squid3 at treenet.co.nz>
>>
>> That is all it needs to do to begin with; parse off the numeric value
>> from the input line and send it back as prefix on the output line. The
> 
>> helper does not need threading or anything particularly special for the > minimal support.
> 
> 
> I thought it had to be asynchronous.
> The docs say "Only used with helpers capable of processing more than one query at a time."
> 
> Example:
> Squid sends "1 URI1" (or whatever) to the helper.
> It does not wait for an immediate response.
> In fact, Squid can send "2 URI2" before getting the reply to ID 1, right?

Yes.


> In my case, the helper is synchronous, non-MT. I don't think it will improve the time responses per-se.
> 
> In any case, my helper won't be able to process more than one query AT A TIME.
> 
> I tried it anyway. So here's the relevant code:
> 
> while( <STDIN> )
> {
> s/^\s*//;
> s/\s*$//;
> my @squidin = split;
> my $squidn = scalar @squidin;
> undef $url;
> undef $channelid;
> if ( ($squidn == 2 ) && (defined $squidin[0]) && ($squidin[0] =~ /^\d+?$/) ) {
> $channelid = $squidin[0];
> $url = $squidin[1] if (defined $squidin[1]);
> } else {
> $url = $squidin[0] if (defined $squidin[0]);
> }
> 
> [...]
> logtofile("Channel-ID: ".$channelid."\n") if ((defined $channelid) && ($debug >= 1));
> 
> [...do DB lookups, reply accordingly...]
> 
> if (defined $channelid) {
> print( $channelid." OK\n" );
> logtofile( $channelid." OK\n" ) if ($debug >= 1);
> } else {
> print( "OK\n" );
> logtofile( "OK\n" ) if ($debug >= 1);
> }
> [...similar responses for ERR messages...]
> }
> 
> Here's the relevant squid.conf line:
> external_acl_type bllookup ttl=86400 negative_ttl=86400 children-max=80 children-startup=10 children-idle=3 concurrency=8 %URI /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl [...]
> 
> How can I check in the Squid log that concurrency is "working"?

Section 82, level 2 or 4 should log the queries.

Better info is in the cachemgr/squidclient "external_acl" report. Each 
helper is listed with its total and summary stats for each helper child.

> 
> If the helper logs to a text file as in the trimmed code above, I notice that the channel ID is always 0. I get messages such as:
> Channel-ID: 0
> 0 OK
> Channel-ID: 0
> 0 ERR ...
> 
> Is this expected?

Maybe.

If you make the helper pause a bit and throw a large number of different 
URLs at Squid you should see it grow a bit higher than 0.

> 
> Despite this, I can see that the number of helper processes does not increase over time for now, and that HTTP/S client browsing is responsive enough.
> # ps aux | grep -c squid_url_lookup.pl
> 11
> 

Yay.

> One last thing. I'm using:
> cache_dir diskd /var/cache/squid 100 16 256
> I may want to try to comment out this directive for improved I/O performance.
> 
> Thanks,
> 
> Vieri

Cheers
Amos