[squid-users] filtering http(s) sites, transparently

[Amos Jeffries]

On 6/04/2016 6:50 a.m., Jok Thuau wrote:
> On Mon, Apr 4, 2016 at 6:23 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:
> 
>>>>>
>>>>> If i remove *all* the http_access lines, then the behavior appears
>>>> correct
>>>>> (from a "splicing/bumping" standpoint).
>>>>>
>>>>
>>>> Strange. Squid without any http_access lines should be denying traffic
>>>> 100%.
>>>>
>>>>
>>> I do not see this behavior. Traffic appears to be allowed, and bumped
>>> (though with the wrong certificate, depending on the config, as explained
>>> before).
>>>
>>>
>>
> 
> 
> 
> 
>>>
>>> my apologies for trying to show only the relevant parts. Find below the
>>> current config.
>>> It appears to be bumping everything rather than splicing any of the
>> config
>>> (which may be due to the limitations documented on the wiki)
>>>
>>> acl Safe_ports port 80 # http
>>> acl Safe_ports port 443 # https
>>> acl SSL_ports port 443
>>> acl CONNECT method CONNECT
>>> http_port 3129 intercept
>>> https_port 8443 intercept ssl-bump generate-host-certificates=on
>>> dynamic_cert_mem_cache_size=64MB \
>>>     cert=/etc/squid/ssl/proxy.pem \
>>>     key=/etc/squid/ssl/proxy.key \
>>>     cafile=/etc/squid/ssl/proxy.pem
>>> always_direct allow all
>>
>> always_direct has not been necessary with SSL-Bump sice 3.1 series. You
>> should remove it.
>>
>>> acl step1 at_step SslBump1
>>> acl step2 at_step SslBump2
>>> acl step3 at_step SslBump3
>>> acl SniBypass ssl::server_name_regex \.slashdot\.org
>>> acl SniBypass ssl::server_name_regex \.fsdn\.com
>>
> 
> I have moved those "SniBypass" acl into a separate files and replaced this
> with an include, as that list will end up growing.
> 

It would be better to add only the list of values to a file. It will
save you entering the ACL name and type repeatedly in the included file.
Like this ...

/some/file/bypass_domains:
 .slashdot.org
 .fsdn.com


/etc/squid/squid.conf:
 ...
 acl http_bypass dstdomain "/some/file/bypass_domains"



> 
>>> acl http_bypass dstdomain .slashdot.org
>>> acl http_bypass dstdomain .fsdn.com
> 
> 
> and similarly here, replaced by an include...
> 
> 
>>
>>> acl https_bypass all-of CONNECT SniBypass
>>
>> This https_bypass ACL definition is a bit weird. It requires a single
>> message to match both TLS and HTTP properties simultaneously.
> 
> 
>> As you might imagine it is difficult for a TLS messages to match HTTP
>> properties, and vice versa. So it wont ever match.
>>
>>
> I don't understand. SniBypass is based on ssl::server_name_regex which
> shouldn't apply to http at all...

Yes.

> Would that not be coming from the (client|server)Hello?
> 

yes.

Also the CONNECT ACL is based on the HTTP method.

For the https_bypass ACL to return true/match the message being tested
must match both ("all-of") CONNECT and SniBypass.

HTTP messages cannot match SniBypass.
TLS messages cannot match CONNECT.
Therefore nothing will ever match all of https_bypass requirements.

> 
>> Note: SNI is *not* equivalent to Host or URL domain name. They can
>> contain very different values. The only thing they have in common is
>> that they both are supposed to point at the IP of the server being
>> contacted.
>>
>>
>>> acl http_ok all-of http_bypass Safe_ports
>>> ssl_bump peek step1
>>> ssl_bump splice SniBypass step2
>>
>> This splice will work if (and only if) the client sends TLS SNI values
>> to Squid. It will ignore the server cert details.
>>
>> For clients which do not send SNI or for all connections where the SNI
>> does not match your ACL the bump rule below will do client-first bumping
>> (without the server cert).
>>
>>> ssl_bump bump all
>>
>> I suggets you try these ssl_bump rules instead:
>> [snip]
> 
>  OK
> 
>> [snip]
>> Okay. That sort of matches your policy. Except that you are missing the
>> security defaults. Those lines are carefully tuned for the specific
>> behaviour to protect against security attacks:
>>
>>  http_access deny !Safe_ports
>>  http_access deny CONNECT !SSL_ports
>>
>> .. and should be above your custom rules.
>>
> 
> I added those at the top as requested...
> 
> 
>>  cache allow all
>>  cache deny all
>>
>> ... pick one.
>>
>>
> done - the deny one is the one left in there now.
> 
>>
>>> shutdown_lifetime 3 seconds
>>
> 
> for clarification, I also moved the two sets of ACLs into separate files,
> as those will eventually be maintained externally (SniBypass and
> http_bypass).
> 
> The config file is now:
> 
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_port 3128
> http_port 3129 intercept
> https_port 8443 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=64MB \
>     cert=/etc/squid/ssl/proxy.pem \
>     key=/etc/squid/ssl/proxy.key \
>     cafile=/etc/squid/ssl/proxy.pem
> workers 6
> always_direct allow all
> acl step1 at_step SslBump1
> acl step2 at_step SslBump2
> acl step3 at_step SslBump3
> include "/etc/squid/snibypass.acl"
> include "/etc/squid/dstbypass.acl"
> acl https_ok all-of CONNECT SniBypass
> acl http_ok all-of http_bypass Safe_ports
> ssl_bump splice SniBypass
> ssl_bump peek step1
> ssl_bump stare step2
> ssl_bump bump all
> sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_ECDH_USE
> sslproxy_cert_sign_hash sha256
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
> sslcrtd_children 8 startup=1 idle=1
> http_access allow http_ok
> http_access allow https_ok
> http_access deny all
> cache deny all
> shutdown_lifetime 3 seconds
> 
> Note that with that config, the http_access deny all (couple lines before
> the end) appears to deny the TLS/SSL connection before the ssl_bump steps
> have a chance to match, so i get certs that are not mimic'ed (they have
> CN=<ip>). If i remove all 3 http_access at the end, then the
> splicing/bumping behavior appears to work as expected, but then i'm not
> denying anything...

You used to have "allow CONNECT" where you now have "allow https_ok".

As mentioned above the https_ok ACL will never match anything because
its sub-ACL tests cannot all produce a true result at the same time.

The "allow CONNECT" was fine at its previous location, and is likely the
best way to keep it there.


> 
> that seems to confirm my suspicion that the access control (http_access)
> apply too early for me to match anything related to the ssl::server_name or
> ssl::server_name_regex.

Both yes and no. The server_name ACL when tested in http_access can
match the host/server name given in a CONNECT message if (and only if)
it actually *is* a name, not a raw-IP address. server_name does not do
DNS lookups like dstdomain.
 For intercepted traffic that field will always be a raw-IP, so
server_name cannot match it against any textual host name.


> 
> I keep thinking that what i'm missing is that the http_access applies too
> early.

You are right so far as that is what is going on. But you are missing an
additional detail as well:

Each of these 'access rule sets are being applied multiple times at
different stages (http_access included).

http_access is being applied on the CONNECT request right at the start
of the processing. And again on the HTTPS bumped traffic after ssl_bump
has finished all its stuff.

> I played with "terminate" instead of "bump" at the last ssl_bump
> command, but i really need the error message. I keep wanting to have
> something like this:
> 
> ssl_bump splice SniBypass
> ssl_bump peek step1
> ssl_bump stare step2
> ssl_bump deny all
> 
> where the last one would effectively bump the connection, and provide the
> ERR_ACCESS_DENIED page.

You cannot emit an HTTP + HTML format denial message in TLS layer protocol.

NP: the 'proper' way to deny TLS layer traffic is with "terminate"
ssl_bump action which sends a TLS layer terminate/abort message. Which
is very different to the HTTP+HTML "Access Denied" page. Namely you do
not get to write anything for the "user" to see.


How all this Squid processing is working is that Squid receives a TCP
connection, synthesizes a CONNECT with raw-IP. Which gets processed by
your http_access rules. That "allow CONNECT" rule at the end of the set
usually matches and lets the bumping begin step1 (unless you decided to
send a denied error page early).

After step1 Squid should have the SNI details for SniBypass to work with
(only inside ssl_bump rules).

So to summarize the sequence is:

-> TCP SYN packets
-> Squid fake CONNECT request:
 http_access --> error page?
 ssl_bump (step 1) --> splice?
 ssl_bump (step 2) --> splice?
 ssl_bump (step 3) --> splice?

-> Some HTTPS request:
 http_access --> error page!


* Only the http_access steps can produce an error page in HTTP.

* The ssl_bump can splice (to accept the tunnel with no error page
possible), or bump (so http_access denying the first HTTPS request
generates the visible access denied error page).

Amos