<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#464646" bgcolor="#FFFFFF">
<font face="Arial">Thanks Elizer !!<br>
<br>
I have tested your code as is in </font><font face="Arial"><font
face="Arial">/lib/squid3/external_acl_first process </font>but
it take 100% CPU and squid freeze requests.<br>
Seems a crazy loop somewhere...<br>
<br>
root 105852 0.0 0.1 73712 9256 ? SNs 00:27 0:00
squid<br>
squid 105854 0.0 0.3 89540 27536 ? SN 00:27 0:00
(squid-1) --kid squid-1<br>
squid 105855 91.6 0.5 219764 47636 ? SNl 00:27 2:52
python /lib/squid3/external_acl_first<br>
squid 105856 91.8 0.5 219768 47672 ? SNl 00:27 2:52
python /lib/squid3/external_acl_first<br>
squid 105857 92.9 0.5 293488 47696 ? SNl 00:27 2:54
python /lib/squid3/external_acl_first<br>
squid 105858 91.8 0.6 367228 49728 ? SNl 00:27 2:52
python /lib/squid3/external_acl_first<br>
</font><br>
I did not find where it should be...<br>
<br>
<br>
<div class="moz-cite-prefix">Le 06/02/2022 à 11:46, Eliezer Croitoru
a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:000101d81b46$d4ca2d70$7e5e8850$@gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
color:#464646;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span style="color:windowtext">Hey David,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Not a fully
completed helper but it seems to works pretty nice and might
be better then what exist already:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><a
href="https://gist.githubusercontent.com/elico/03938e3a796c53f7c925872bade78195/raw/21ff1bbc0cf3d91719db27d9d027652e8bd3de4e/threaded-helper-example.py"
moz-do-not-send="true" class="moz-txt-link-freetext">https://gist.githubusercontent.com/elico/03938e3a796c53f7c925872bade78195/raw/21ff1bbc0cf3d91719db27d9d027652e8bd3de4e/threaded-helper-example.py</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">#!/usr/bin/env python<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">import sys<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">import time<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">import urllib.request<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">import signal<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">import threading<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">#set debug mode for True or
False<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">debug = False<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">#debug = True<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">queue = []<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">threads = []<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">RUNNING = True<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">quit = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">rand_api_url =
<a class="moz-txt-link-rfc2396E" href="https://cloud1.ngtech.co.il/api/test.php">"https://cloud1.ngtech.co.il/api/test.php"</a><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">def sig_handler(signum, frame):<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> sys.stderr.write("Signal is
received:" + str(signum) + "\n")<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> global quit<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> quit = 1<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> global RUNNING<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> RUNNING=False<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">def handle_line(line):<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> if not RUNNING:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> return<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> if not line:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> return<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> if quit > 0:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> return<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> arr = line.split()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> response =
urllib.request.urlopen( rand_api_url )<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> response_text =
response.read()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> queue.append(arr[0] + " " +
response_text.decode("utf-8"))<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">def handle_stdout(n):<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> while RUNNING:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> if quit > 0:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> return<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> while len(queue) >
0:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> item = queue.pop(0)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">
sys.stdout.write(item)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> sys.stdout.flush()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> time.sleep(0.5)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">def handle_stdin(n):<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> while RUNNING:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> line =
sys.stdin.readline()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> if not line:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> break<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> if quit > 0:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> break<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> line = line.strip()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> thread =
threading.Thread(target=handle_line, args=(line,))<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> thread.start()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> threads.append(thread)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">signal.signal(signal.SIGUSR1,
sig_handler)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">signal.signal(signal.SIGUSR2,
sig_handler)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">signal.signal(signal.SIGALRM,
sig_handler)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">signal.signal(signal.SIGINT,
sig_handler)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">signal.signal(signal.SIGQUIT,
sig_handler)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">signal.signal(signal.SIGTERM,
sig_handler)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">stdout_thread =
threading.Thread(target=handle_stdout, args=(1,))<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">stdout_thread.start()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">threads.append(stdout_thread)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">stdin_thread =
threading.Thread(target=handle_stdin, args=(2,))<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">stdin_thread.start()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">threads.append(stdin_thread)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">while(RUNNING):<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> time.sleep(3)<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">print("Not RUNNING")<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">for thread in threads:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext"> thread.join()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Courier
New";color:windowtext">print("All threads stopped.")<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">## END<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Eliezer<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:windowtext">----<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Eliezer
Croitoru<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">NgTech,
Tech Support<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Mobile:
+972-5-28704261<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Email: <a
href="mailto:ngtech1ltd@gmail.com"
moz-do-not-send="true" class="moz-txt-link-freetext">ngtech1ltd@gmail.com</a><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:windowtext">From:</span></b><span
style="color:windowtext"> squid-users
<a class="moz-txt-link-rfc2396E" href="mailto:squid-users-bounces@lists.squid-cache.org"><squid-users-bounces@lists.squid-cache.org></a> <b>On
Behalf Of </b>David Touzeau<br>
<b>Sent:</b> Friday, February 4, 2022 16:29<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:squid-users@lists.squid-cache.org">squid-users@lists.squid-cache.org</a><br>
<b>Subject:</b> Re: [squid-users] external helper
development<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Elizer,<br>
<br>
Thanks for all this advice and indeed your arguments are valid
between opening a socket, sending data, receiving data and
closing the socket unlike direct access to a regex or a memory
entry even if the calculation has already been done.<br>
<br>
But what surprises me the most is that we have produced a
python plugin in thread which I provide you a code below. <br>
The php code is like your mentioned example ( No thread, just
a loop and output OK ) <br>
<br>
Results are after 6k requests, squid freeze and no surf can be
made as with PHP code we can up to 10K requests and squid is
happy<br>
really, we did not understand why python is so low.<br>
<br>
<span style="font-family:"Arial",sans-serif">Here a
python code using threads<br>
<br>
#!/usr/bin/env python<br>
import os<br>
import sys<br>
import time<br>
import signal<br>
import locale<br>
import traceback<br>
import threading<br>
import select<br>
import traceback as tb<br>
<br>
class ClienThread():<br>
<br>
def __init__(self):<br>
self._exiting = False<br>
self._cache = {}<br>
<br>
def exit(self):<br>
self._exiting = True<br>
<br>
def stdout(self, lineToSend):<br>
try:<br>
sys.stdout.write(lineToSend)<br>
sys.stdout.flush()<br>
<br>
except IOError as e:<br>
if e.errno==32:<br>
# Error Broken PIPE!"<br>
pass<br>
except:<br>
# other execpt<br>
pass<br>
<br>
def run(self):<br>
while not self._exiting:<br>
if sys.stdin in select.select([sys.stdin], [],
[], 0.5)[0]:<br>
line = sys.stdin.readline()<br>
LenOfline=len(line)<br>
<br>
if LenOfline==0:<br>
self._exiting=True<br>
break<br>
<br>
if line[-1] == '\n':line = line[:-1]<br>
channel = None<br>
options = line.split()<br>
<br>
try:<br>
if options[0].isdigit(): channel =
options.pop(0)<br>
except IndexError:<br>
self.stdout("0 OK first=ERROR\n")<br>
continue<br>
<br>
# Processing here<br>
<br>
try:<br>
self.stdout("%s OK\n" % channel)<br>
except:<br>
self.stdout("%s ERROR first=ERROR\n" %
channel)<br>
<br>
<br>
<br>
<br>
class Main(object):<br>
def __init__(self):<br>
self._threads = []<br>
self._exiting = False<br>
self._reload = False<br>
self._config = ""<br>
<br>
for sig, action in (<br>
(signal.SIGINT, self.shutdown),<br>
(signal.SIGQUIT, self.shutdown),<br>
(signal.SIGTERM, self.shutdown),<br>
(signal.SIGHUP, lambda s, f: setattr(self,
'_reload', True)),<br>
(signal.SIGPIPE, signal.SIG_IGN),<br>
):<br>
try:<br>
signal.signal(sig, action)<br>
except AttributeError:<br>
pass<br>
<br>
<br>
<br>
def shutdown(self, sig = None, frame = None):<br>
self._exiting = True<br>
self.stop_threads()<br>
<br>
def start_threads(self):<br>
<br>
sThread = ClienThread()<br>
t = threading.Thread(target = sThread.run)<br>
t.start()<br>
self._threads.append((sThread, t))<br>
<br>
<br>
<br>
def stop_threads(self):<br>
for p, t in self._threads:<br>
p.exit()<br>
for p, t in self._threads:<br>
t.join(timeout = 1.0)<br>
self._threads = []<br>
<br>
def run(self):<br>
""" main loop """<br>
ret = 0<br>
self.start_threads()<br>
return ret<br>
<br>
<br>
if __name__ == '__main__':<br>
# set C locale<br>
locale.setlocale(locale.LC_ALL, 'C')<br>
os.environ['LANG'] = 'C'<br>
ret = 0<br>
try:<br>
main = Main()<br>
ret = main.run()<br>
except SystemExit:<br>
pass<br>
except KeyboardInterrupt:<br>
ret = 4<br>
except:<br>
sys.exit(ret)</span><o:p></o:p></p>
<div>
<p class="MsoNormal">Le 04/02/2022 à 07:06, Eliezer Croitoru a
écrit :<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">And about the cache of each
helpers, the cost of a cache on a single helper is not
much in terms of memory comparing to some network access.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">Again it’s possible to test and
verify this on a loaded system to get results. The delay
itself can be seen from squid side in the cache manager
statistics.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">You can also try to compare the
next ruby helper:</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext"><a
href="https://wiki.squid-cache.org/EliezerCroitoru/SessionHelper"
moz-do-not-send="true" class="moz-txt-link-freetext">https://wiki.squid-cache.org/EliezerCroitoru/SessionHelper</a></span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">About a shared “base” which
allows helpers to avoid computation of the query…. It’s a
good argument, however it depends what is the cost of<br>
pulling from the cache compared to calculating the answer.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">A very simple string comparison
or regex matching would probably be faster than reaching a
shared storage in many cases.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">Also take into account the
“concurrency” support from the helper side.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">A helper that supports parallel
processing of requests/lines can do better then many
single helpers in more than once use case.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">In any case I would suggest to
enable requests concurrency from squid side since the
STDIN buffer will emulate some level of concurrency<br>
by itself and will allow squid to keep going forward
faster.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">Just to mention that SquidGuard
have used a single helper cache for a very long time, ie
every single SquidGuard helper has it’s own copy of the
whole</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">configuration and database files
in memory.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">And again, if you do have any
option to implement a server service model and that the
helpers will contact this main service you will be able to
implement<br>
much faster internal in-memory cache compared to a
redis/memcahe/other external daemon(need to be tested).</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">A good example for this is
ufdbguard which has helpers that are clients of the main
service which does the whole heavy lifting and also holds
<br>
one copy of the DB.</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="color:windowtext">I have implemented SquidBlocker
this way and have seen that it out-performs any other
service I have tried until now.</span><o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
<br>
</body>
</html>