[squid-users] Squid proxy server to log HTTPS traffic

[Helen Rai]

HI,

Thanks, will try and update.

----
Helen Rai
Platform operations, Nepallink
44260822 || 9841262275 || support at nepallink.net <abuse at nepallink.net>
http://www.nepallink.net || Link Road, Khusibu
<http://www.facebook.com/pages/NepalLink-Network/334213043398796>
<http://twitter.com/nepallink>

On Thu, Aug 2, 2018 at 4:29 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:

> On 02/08/18 19:05, Helen Rai wrote:
> > Now what I want to do is log all the web traffic that is requested from
> > those devices which are using WiFi.
> > I have attached the configuration of squid and Mikrotik which I had done
> > for HTTP. (Please note that when I had done this for HTTP, the squid
> > wasn't configured with --enable-SSL or --enable-crtd)
> >
>
> Yes, those options are required to handle the TLS which is part of
> HTTPS. HTTP is not encrypted. Or rather: --with-openssl is required
>
> Since you are rebuilding it with these extra options I recommend using
> the Squid-4 code from the Debian Sid repository if you can. It contains
> much better handling for HTTPS traffic which will be useful later to
> avoid non-HTTPS traffic arriving on port 443.
>
>
> > I had referred some online sites to do for HTTPS but none of them
> > worked. Please help if anyone knows how to log all HTTPS request.
> >
> > I have attached document regarding the rules applied in Mikrotik and
> > squid3 configuration for HTTP.
> >
>
> Okay. So first thing to do is fix some mistakes in your squid.conf
>
> * "transparent" has long ago been replaced by "intercept"
>
>  http_port 3127 intercept
>
> * remove incomplete / broken config line
>
>  #http_access deny
>
> * remove the first two of these repeated lines:
>
>  http_access allow localhost
>  ...
>  http_access allow localhost
>  http_access allow localhost
>  ...
>
> * stop using "localhost" as the ACL name to represent Department-A.
> Localhost is a reserved term in networking with a very specific meaning
> (different to what you are using it for) which is also baked into Squid
> default configuration settings.
>
> Instead use an ACL which names the thing being matched. For example:
>
>  acl Department-A src 10.0.7.0/24
>
>
> * move your custom setting down below the default security http_access
> rules. Those are the deny lines referring to SSL_ports and Safe_ports.
>
> Your http_access lines should now look nice and simple, like this:
>
>
>  http_access deny !Safe_ports
>  http_access deny CONNECT !SSL_ports
>
>  acl test dstdomain "/etc/squid3/acl/test"
>  http_access deny test
>
>  acl Department-A src 10.0.7.0/24
>  http_access allow Department-A
>
>  http_access deny all
>
>
> *** Your HTTP traffic should still be fully operational and doing what
> you want after these changes. If not something is broken we need to fix
> before continuing.
>
>
> Second, add the pieces necessary for handling port 443 traffic to
> squid.conf.
>
> You *will* need to generate a self-signed CA certificate with the
> signing-cert (CA) and and HTTP server properties to use SSL-Bump features.
>   But that will only need to be installed on clients machine *if* you go
> as far as decrypting the HTTPS traffic or producing errors for clients.
> This is where Squid-4 handles TLS far more cleanly than Squid-3.5.
>
>
>  # A port to receive port 443 traffic from Mikrotik
>  https_port 3129 intercept ssl-bump cert=/etc/squid3/your-CA.pem
>
>  ... note the 's' in that directive name. A typo there breaks HTTPS in
> confusing ways.
>
>
>  # SSL-Bump rules to peek at details for logging, then relay the traffic
>  acl step1 at_step SslBump1
>  ssl_bump peek step1
>  ssl_bump splice all
>
>  # Squid-4 can pass any non-HTTPS through or omit the below line to
> deliver error pages signed by the your-CA.pem certificate.
>
>   on_unsupported_protocol tunnel
>
>
> Thirdly, you need to add an entry to the Raspberry Pi NAT rules. Same as
> for the port 80 NAT rule that should also exist, but for port 443.
>
>  ** at this point you should be able to test the Raspberry PI is working
> for HTTPS by setting a test machine to have the R-Pi IP address as its
> gateway and requesting any https:// URL in a Browser, or wget, curl,
> squidclient etc.
>
>
> Lastly, when Squid is happily running with all that add to your Mikrotik
> config a second line identical to the first one in your attachment -
> except that it ends with "dst-port=443".
>
>  As soon as that line is active in your router the HTTPS traffic should
> start arriving and being logged at the Squid.
>
>
>
> Please be aware of the following details:
>
> * Do expect to see at least half of HTTPS traffic being CONNECT with
> raw-IPs and 0 bytes transferred. That is from the first step of
> SSL-Bump'ing. They should be followed by another CONNECT with the TLS
> SNI domain name and correct size of data transferred (one-way, to client).
>
>
> * The SNI is only accurate for honest clients. It can be forged and the
> real HTTPS messages be using another domain name hosted on the same
> server/CDN network.
>  However, to be properly sure that the traffic inside the TLS is going
> to the domain the SNI claimed you will have to fully decrypt the traffic
> (bump action) and that means installing your CA cert on all clients
> machines. Plus configuring dynamic cert generation on the https_port line.
>  Start with getting the above non-decrypt setup working before going
> near bumping.
>
>
> Amos
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180803/e6c59dfa/attachment.html>