[squid-users] Large text ACL lists

[Darren]

Hi

I have now opened the Pandora box of writing my own helper as per Bobs suggestion. 

I am playing with the idea of pre-processing my acl lists and using memcached as a KV store. This way I should be able to update ACL members whilst keeping everything as available as possible.

I would update the acl members outside of squid so it should be fast, and if I get my tree model right, fast and scale well too.

I have had great success with Memcache on various large web applications so again, pending a clever tree algorithm, this could provide me with what I need without having to reload / restart squid.

Darren B.






Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm_campaign=sent-from-mailbird]
On 2/10/2016 12:15:52 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:
On 2/10/2016 2:08 p.m., Benjamin E. Nichols wrote:
> I wouldnt advise reconfigure for when you update your blacklists, sure
> it sounds great, but in reality, as I said, in my experience, only
> sometimes will it actually reload the acl from disk, sometimes it wont.
> Youll do a reconfigure and discover your squid is still running the old
> acls which presumably are memory resident, now this may have been a bug
> thats been since fixed, but Im not messing with it. In our testig
> environment we dont have time to be dealing with squid deciding to load
> an acl from disk when it feels like it should do so, we need it to load
> from disk every time.
>
> Once you do your own testing youll see what im talking about, go ahead
> and add some urls to your acl and -k reconfigure, do this a few times,

So you are sening Squid a series of reload signals so fast it does not
have time to complete one before the next is arriving?

There are many fixes in Squid-4 and latest 3.5 for those situations. But
still some open bug reports about the behaviour there. Those are not
related to ACLs specifically. Any reconfig task that takes longer than
the time between -k reconfigure signals being sent will trigger issues.

... Ironically using a helper is one of the things which breaks. Squid
looses track of whether any given new helper being started was for the
current or previous -k reconfigure signal.

And of course connections and transactions which are already underway
are not affected by newly loaded config details.

Amos

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20161002/5064f49d/attachment.html>