[squid-users] url_rewrite_program and ACLs

[Amos Jeffries]

On 09/11/17 20:51, Vieri wrote:
> 
> ________________________________
> From: Amos Jeffries
>>
>> acl foo ...
>>   http_access deny foo
>>   deny_info 302:http://example.com/ foo
>>
>> In Squid-3.2+ the deny_info URL portion can use logformat macros for
>> dynamic redirection - like the "rew" substitutions only changing
>> portions of the URL.
> 
> 
> I was already using deny_info like this:
> 
> deny_info http://squidserver/proxy-error/?a=%a&B=%B&e=%e&E=%E&H=%H&i=%i&M=%M&o=%o&R=%R&T=%T&U=%U&u=%u&w=%w&x=%x&acl=denied_domains denied_domains
> 
> I was wondering how to do an immediate redirect without doing it from my custom php script. Now I see.
> 
> I guess I'll have trouble redirecting https sites though... (TLS/SSL trust issues)

The proper HTTP *redirect* with 3xx statsu code has no problems there. 
Since the redirect is simply telling the client to connect elsewhere.

It is re-writing the URL which encounters cert problems because the 
server gets secretly changed from what the client "knows" it is 
connected to mid-way along the connection.

If you are redirecting https:// URLs to a http:// location they may 
still warn about insecure content. But that is not a cert issue, just 
the browser scare tactics for pushing its "TLS-Everywhere" policy on 
your users.


> 
> I don't know if I can "cleanly" redirect from an HTTPS to an HTTP site (ie. so the user's browser doesn't show a "can't open page" message of some sort...).
> 
> You mention that I can avoid using SG, ufdbGuard, or any other redirector/helper for access control. The problem I see when trying to use huge plain text blacklists within Squid directly is that it takes a LOT of time for the proxy cache to start up as it populates the ACLs.
> I can't afford to wait for Squid to do that before serving client requests. I'd rather "allow everything" until the ACLs are populated than have users wait for so long.
> Am I missing something? Is there a way to tell Squid to process the ACLs "in the background", but start handling requests immediately. If so, is it also possible to tell squid in which order to process the ACLs, ie. first process the allowed_domains ACL, and then the denied_domains ACLs? (is ordering in squid.conf enough?)

Darn. You have the one case that calls for keeping the helper :-(

You can still move the ACLs that load in a reasonable times into 
squid.conf and leave the others in SG/ufdbguard. Using 
url_rewrite_access to restrict which transactions the helper gets 
involved with. That will reduce its latency impact on lie traffic, but 
still cause much the same memory related (non-)issues as now.

> 
> I'm saying this because I've sometimes had the need to restart Squid during the worst time of the day (peak working hours). That usually happens when I have an issue with too many open file descriptors (working on it). Stopping it cleanly takes up to 2-3 minutes. If I had to wait several more minutes for Squid to start again because it re-populates huge blacklist ACLs then I think they'd hang me for it.
> 

One thing that may also help you there while you figure out what those 
FD issues are caused by:

If you run "squid -k shutdown" once it signals a slow graceful shutdown 
of Squid. Which will take a *minimum* of the time configured in 
shutdown_lifetime directive (default 30sec) to wait for existing clients 
to finish up and disconnect.

Running "squid -k shutdown" a _second_ time sends the running proxy a 
signal to immediately skip to the processing as if the shutdown_lifetime 
had already been reached.

The shutdown time with two sequential calls should be only the time 
needed to close all the open FD connections, shutdown helpers and 
release however much memory Squid is using. All that happens in just a 
few seconds normally, but I'm not sure if "normal" matches your FD 
overload case. Will definitely be faster than waiting for the graceful 
shutdown anyhow.


Amos