[squid-users] Help: squid restarts and squidGuard die

[Marcus Kool]

On 20/09/18 08:46, Amos Jeffries wrote:
> On 19/09/18 11:49 PM, Marcus Kool wrote:
>>
>> On 18/09/18 23:03, Amos Jeffries wrote:
>>> On 19/09/18 1:54 AM, neok wrote:
>>>> Thank you very much Amos for putting me in the right direction.
>>>> I successfully carried out the modifications you indicated to me.
>>>> Regarding ufdbGuard, if I understood correctly, what you recommend is
>>>> to use
>>>> the ufdbConvertDB tool to convert my blacklists in plain text to the
>>>> ufdbGuard database format? And then use that/those databases in
>>>> normal squid
>>>> ACL's?
>>>
>>> No, ufdbguard is a fork of SquidGuard that can be used as a drop-in
>>> replacement which works better while you improve your config.
>>>
>>> You should work towards less complexity. Squid / squid.conf is where
>>> HTTP access control takes place. The helper is about re-writing the URL
>>> (only) - which is a complex and destructive process.
>>
>> ufdbGuard is a simple tool that has the same syntax in its configuration
>> file as squidGuard has.
>> It is far from complex, has a great Reference Manual, exmaple config
>> file and a responsive support desk.
>> Amos, I have never seen you calling a URL writer being a complex and
>> destructive process.  What do you mean?
> 
> Re-writing requires Squid to:
>   * fork external helpers, and
>   * maintain queues of lookups to those helpers, and
>   * maintain cache of helper responses, and
>   * maintain a whole extra copy of HTTP-request state, and
>   * copy some (not all) of that state info between the two "client" requests.
> 
>   ... lots of complexity, memory, CPU time, traffic latency, etc.

Squid itself is complex and for any feature of Squid one can make a list like above to say that it is complex.
The fact that one can make such a list does not mean much to me.
One can make the same or a similar list for external acl helpers and even native acls.

> Also when used for access control (re-write to an "error" URL) the
> re-write helper needs extra complexity in itself to act as the altered
> origin server for error pages, or have some fourth-party web server.

Squid cannot do everything that a URL writer, and specifically ufdbGuard, can.
For example, Squid must restart and break all open connections when a tiny detail of the configuration changes.  With ufdbGuard this does not happen.
ufdbGuard supports dynamic lists of users, domains and source ip addresses which are updated every X minutes without any service interruption.
When other parameters change, ufdbGuard resets itself with zero service interruption for Squid and its users.
ufdbGuard can decide to probe a site to make a decision, and hence detect Skype, Teamviewer and other types of sites that an admin might want to block.  Squid cannot.
ufdbGuard can decide to do a lookup of a reverse IP lookup to make a decision.  Squid cannot.
ufdbGuard supports complex time restrictions for access. Squid support simple time restrictions.
ufdbGuard supports flat file domain/url lists and a commercial URL database.  Squid does not.
And the list goes on.

So when you state on the mailing list that users should unconditionally stop using a URL writer in favor of using Squid acls, you may be causing troubles for admins who do not know the implications of 
your advice.


>> URL rewriters have been used for decades for HTTP access control but you
>> state "squid.conf is where HTTP access control takes place".
> 
> Once upon a time, back at the dawn of the WWW (before the 1990s) Squid
> lacked external_acl_type and modular ACLs.
> 
> That persisted for the first decade or so of Squid's life, with only the
> re-write API for admin to use for complicated permissions.
> 
> Then one day about 2 decades or so ago, external ACL was added and the
> ACLs were also made much easier to implement and plug in new checks.
> Today we have hundreds of native ACLs and even a selection of custom ACL
> helpers. Making the need for these abuses of the poor re-writers.
> 
> Old habits and online tutorials however are hard to get rid of.

If you want to get rid of habits that in your view are old/obsolete, then why not start a discussion?
And in the event that at the end of the discussion, the decision is made that a particular interface should be removed, why not phase it out ?

>> Are you saying that you want it is the _only_ place for HTTP access
>> control?
> 
> 
> I'm saying the purpose of the url_rewrite_* API in Squid is to tell
> Squid whether the URL (only) needs some mangling in order for the
> server/origin to understand it.
>   It can re-write transparently with all the problems that causes to
> security scopes and URL sync between the endpoints. Or redirect the
> client to the "correct" URL.
> 
> 
> The Squid http_access and similar *access controls* are the place for
> access control - hint is in the naming. With external ACL type for
> anything Squid does not support natively or well. As Flashdown mentioned
> even calls to SquidGuard etc. can be wrapped and used as external ACLs.

Wrapping and externals ACLs adds the same complexity, memory, CPU time, traffic latency, etc that you use as an argument against a URL writer.
Is it only because of the name, 'external acl helper' vs 'url rewriter helper', that you dislike the url rewriter?

Marcus

> 
> Amos