[squid-users] squid cache takes a break

Eliezer Croitoru eliezer at ngtech.co.il
Tue Sep 12 20:51:36 UTC 2017


I just must add that if you understand how TCP works(which the helpers use to communicate with squid) then it makes sense that it is possible that...
The sender (ie squid) sent 100 lines but the client software yet to process them since it's in the OS or other software\hardware related buffer.

For me it was hard to understand at first since it's an STDIN\STDOUT interface so it would block after every write but it's not...

There is a possibility that if the helper can process every incoming request with threading or other method of concurrency then the performance of the helper and by that squid will be better but if only using the basic buffer works fine for you then great.

Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il



-----Original Message-----
From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Amos Jeffries
Sent: Tuesday, September 12, 2017 16:08
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] squid cache takes a break

On 12/09/17 22:59, Vieri wrote:
> 
> ________________________________
> From: Amos Jeffries <squid3 at treenet.co.nz>
>>
>> That is all it needs to do to begin with; parse off the numeric value
>> from the input line and send it back as prefix on the output line. The
> 
>> helper does not need threading or anything particularly special for the > minimal support.
> 
> 
> I thought it had to be asynchronous.
> The docs say "Only used with helpers capable of processing more than one query at a time."
> 
> Example:
> Squid sends "1 URI1" (or whatever) to the helper.
> It does not wait for an immediate response.
> In fact, Squid can send "2 URI2" before getting the reply to ID 1, right?

Yes.


> In my case, the helper is synchronous, non-MT. I don't think it will improve the time responses per-se.
> 
> In any case, my helper won't be able to process more than one query AT A TIME.
> 
> I tried it anyway. So here's the relevant code:
> 
> while( <STDIN> )
> {
> s/^\s*//;
> s/\s*$//;
> my @squidin = split;
> my $squidn = scalar @squidin;
> undef $url;
> undef $channelid;
> if ( ($squidn == 2 ) && (defined $squidin[0]) && ($squidin[0] =~ /^\d+?$/) ) {
> $channelid = $squidin[0];
> $url = $squidin[1] if (defined $squidin[1]);
> } else {
> $url = $squidin[0] if (defined $squidin[0]);
> }
> 
> [...]
> logtofile("Channel-ID: ".$channelid."\n") if ((defined $channelid) && ($debug >= 1));
> 
> [...do DB lookups, reply accordingly...]
> 
> if (defined $channelid) {
> print( $channelid." OK\n" );
> logtofile( $channelid." OK\n" ) if ($debug >= 1);
> } else {
> print( "OK\n" );
> logtofile( "OK\n" ) if ($debug >= 1);
> }
> [...similar responses for ERR messages...]
> }
> 
> Here's the relevant squid.conf line:
> external_acl_type bllookup ttl=86400 negative_ttl=86400 children-max=80 children-startup=10 children-idle=3 concurrency=8 %URI /opt/custom/scripts/run/scripts/firewall/squid_url_lookup.pl [...]
> 
> How can I check in the Squid log that concurrency is "working"?

Section 82, level 2 or 4 should log the queries.

Better info is in the cachemgr/squidclient "external_acl" report. Each 
helper is listed with its total and summary stats for each helper child.

> 
> If the helper logs to a text file as in the trimmed code above, I notice that the channel ID is always 0. I get messages such as:
> Channel-ID: 0
> 0 OK
> Channel-ID: 0
> 0 ERR ...
> 
> Is this expected?

Maybe.

If you make the helper pause a bit and throw a large number of different 
URLs at Squid you should see it grow a bit higher than 0.

> 
> Despite this, I can see that the number of helper processes does not increase over time for now, and that HTTP/S client browsing is responsive enough.
> # ps aux | grep -c squid_url_lookup.pl
> 11
> 

Yay.

> One last thing. I'm using:
> cache_dir diskd /var/cache/squid 100 16 256
> I may want to try to comment out this directive for improved I/O performance.
> 
> Thanks,
> 
> Vieri

Cheers
Amos
_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users



More information about the squid-users mailing list