[squid-users] filtering http(s) sites, transparently
squid3 at treenet.co.nz
Wed Apr 6 11:32:19 UTC 2016
On 6/04/2016 6:50 a.m., Jok Thuau wrote:
> On Mon, Apr 4, 2016 at 6:23 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:
>>>>> If i remove *all* the http_access lines, then the behavior appears
>>>>> (from a "splicing/bumping" standpoint).
>>>> Strange. Squid without any http_access lines should be denying traffic
>>> I do not see this behavior. Traffic appears to be allowed, and bumped
>>> (though with the wrong certificate, depending on the config, as explained
>>> my apologies for trying to show only the relevant parts. Find below the
>>> current config.
>>> It appears to be bumping everything rather than splicing any of the
>>> (which may be due to the limitations documented on the wiki)
>>> acl Safe_ports port 80 # http
>>> acl Safe_ports port 443 # https
>>> acl SSL_ports port 443
>>> acl CONNECT method CONNECT
>>> http_port 3129 intercept
>>> https_port 8443 intercept ssl-bump generate-host-certificates=on
>>> dynamic_cert_mem_cache_size=64MB \
>>> cert=/etc/squid/ssl/proxy.pem \
>>> key=/etc/squid/ssl/proxy.key \
>>> always_direct allow all
>> always_direct has not been necessary with SSL-Bump sice 3.1 series. You
>> should remove it.
>>> acl step1 at_step SslBump1
>>> acl step2 at_step SslBump2
>>> acl step3 at_step SslBump3
>>> acl SniBypass ssl::server_name_regex \.slashdot\.org
>>> acl SniBypass ssl::server_name_regex \.fsdn\.com
> I have moved those "SniBypass" acl into a separate files and replaced this
> with an include, as that list will end up growing.
It would be better to add only the list of values to a file. It will
save you entering the ACL name and type repeatedly in the included file.
Like this ...
acl http_bypass dstdomain "/some/file/bypass_domains"
>>> acl http_bypass dstdomain .slashdot.org
>>> acl http_bypass dstdomain .fsdn.com
> and similarly here, replaced by an include...
>>> acl https_bypass all-of CONNECT SniBypass
>> This https_bypass ACL definition is a bit weird. It requires a single
>> message to match both TLS and HTTP properties simultaneously.
>> As you might imagine it is difficult for a TLS messages to match HTTP
>> properties, and vice versa. So it wont ever match.
> I don't understand. SniBypass is based on ssl::server_name_regex which
> shouldn't apply to http at all...
> Would that not be coming from the (client|server)Hello?
Also the CONNECT ACL is based on the HTTP method.
For the https_bypass ACL to return true/match the message being tested
must match both ("all-of") CONNECT and SniBypass.
HTTP messages cannot match SniBypass.
TLS messages cannot match CONNECT.
Therefore nothing will ever match all of https_bypass requirements.
>> Note: SNI is *not* equivalent to Host or URL domain name. They can
>> contain very different values. The only thing they have in common is
>> that they both are supposed to point at the IP of the server being
>>> acl http_ok all-of http_bypass Safe_ports
>>> ssl_bump peek step1
>>> ssl_bump splice SniBypass step2
>> This splice will work if (and only if) the client sends TLS SNI values
>> to Squid. It will ignore the server cert details.
>> For clients which do not send SNI or for all connections where the SNI
>> does not match your ACL the bump rule below will do client-first bumping
>> (without the server cert).
>>> ssl_bump bump all
>> I suggets you try these ssl_bump rules instead:
>> Okay. That sort of matches your policy. Except that you are missing the
>> security defaults. Those lines are carefully tuned for the specific
>> behaviour to protect against security attacks:
>> http_access deny !Safe_ports
>> http_access deny CONNECT !SSL_ports
>> .. and should be above your custom rules.
> I added those at the top as requested...
>> cache allow all
>> cache deny all
>> ... pick one.
> done - the deny one is the one left in there now.
>>> shutdown_lifetime 3 seconds
> for clarification, I also moved the two sets of ACLs into separate files,
> as those will eventually be maintained externally (SniBypass and
> The config file is now:
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_port 3128
> http_port 3129 intercept
> https_port 8443 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=64MB \
> cert=/etc/squid/ssl/proxy.pem \
> key=/etc/squid/ssl/proxy.key \
> workers 6
> always_direct allow all
> acl step1 at_step SslBump1
> acl step2 at_step SslBump2
> acl step3 at_step SslBump3
> include "/etc/squid/snibypass.acl"
> include "/etc/squid/dstbypass.acl"
> acl https_ok all-of CONNECT SniBypass
> acl http_ok all-of http_bypass Safe_ports
> ssl_bump splice SniBypass
> ssl_bump peek step1
> ssl_bump stare step2
> ssl_bump bump all
> sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_ECDH_USE
> sslproxy_cert_sign_hash sha256
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
> sslcrtd_children 8 startup=1 idle=1
> http_access allow http_ok
> http_access allow https_ok
> http_access deny all
> cache deny all
> shutdown_lifetime 3 seconds
> Note that with that config, the http_access deny all (couple lines before
> the end) appears to deny the TLS/SSL connection before the ssl_bump steps
> have a chance to match, so i get certs that are not mimic'ed (they have
> CN=<ip>). If i remove all 3 http_access at the end, then the
> splicing/bumping behavior appears to work as expected, but then i'm not
> denying anything...
You used to have "allow CONNECT" where you now have "allow https_ok".
As mentioned above the https_ok ACL will never match anything because
its sub-ACL tests cannot all produce a true result at the same time.
The "allow CONNECT" was fine at its previous location, and is likely the
best way to keep it there.
> that seems to confirm my suspicion that the access control (http_access)
> apply too early for me to match anything related to the ssl::server_name or
Both yes and no. The server_name ACL when tested in http_access can
match the host/server name given in a CONNECT message if (and only if)
it actually *is* a name, not a raw-IP address. server_name does not do
DNS lookups like dstdomain.
For intercepted traffic that field will always be a raw-IP, so
server_name cannot match it against any textual host name.
> I keep thinking that what i'm missing is that the http_access applies too
You are right so far as that is what is going on. But you are missing an
additional detail as well:
Each of these 'access rule sets are being applied multiple times at
different stages (http_access included).
http_access is being applied on the CONNECT request right at the start
of the processing. And again on the HTTPS bumped traffic after ssl_bump
has finished all its stuff.
> I played with "terminate" instead of "bump" at the last ssl_bump
> command, but i really need the error message. I keep wanting to have
> something like this:
> ssl_bump splice SniBypass
> ssl_bump peek step1
> ssl_bump stare step2
> ssl_bump deny all
> where the last one would effectively bump the connection, and provide the
> ERR_ACCESS_DENIED page.
You cannot emit an HTTP + HTML format denial message in TLS layer protocol.
NP: the 'proper' way to deny TLS layer traffic is with "terminate"
ssl_bump action which sends a TLS layer terminate/abort message. Which
is very different to the HTTP+HTML "Access Denied" page. Namely you do
not get to write anything for the "user" to see.
How all this Squid processing is working is that Squid receives a TCP
connection, synthesizes a CONNECT with raw-IP. Which gets processed by
your http_access rules. That "allow CONNECT" rule at the end of the set
usually matches and lets the bumping begin step1 (unless you decided to
send a denied error page early).
After step1 Squid should have the SNI details for SniBypass to work with
(only inside ssl_bump rules).
So to summarize the sequence is:
-> TCP SYN packets
-> Squid fake CONNECT request:
http_access --> error page?
ssl_bump (step 1) --> splice?
ssl_bump (step 2) --> splice?
ssl_bump (step 3) --> splice?
-> Some HTTPS request:
http_access --> error page!
* Only the http_access steps can produce an error page in HTTP.
* The ssl_bump can splice (to accept the tunnel with no error page
possible), or bump (so http_access denying the first HTTPS request
generates the visible access denied error page).
More information about the squid-users