[squid-users] https log message formatting help

[Amos Jeffries]

On 5/04/2017 6:00 p.m., daveh wrote:
> Hi squid users
> 
> Is there any way to change the request url log format for HTTPS messages?
> 
> I am using %ru to pull out the URL. When we get https connections, we see
> the url logged as www.microsoft.com:443 

You are assumping that URI means HTTPS. It may seem reasonable, but is
wrong.

The CONNECT request is a _tunnel_ request. It is an opaque *TCP* tunnel.

There is no guarantee that any given port-443 tunnel request is actually
HTTPS these days. There is WebSockets, SPDY, HTTP/2, and a number of
custom protocols inside TLS, and non-TLS protocols as well all using the
port.

When HTTPS does go through a port-443 tunnel, there is often more than
one HTTPS request. So writing https://blah/ to the log would be a lie,
and a deceptive one at that.


> 
> is there any way to reformat the log message to remove the appended port?


Well, the log %ru code is intended to record the *actual* details being
received. What you are seeing is what actually exists in the traffic.

It is a URI type called "authority-form".
<https://tools.ietf.org/html/rfc7230#section-5.3.3>

There is no protocol scheme, no path, no query and no fragment portions
for Squid to work with.



> to go further and rewrite to use https://<url>?

You can always define a log format that prints out the pieces of the URI
as separate format components "%>rs://%>rd:%>rP%>rp"
<http://www.squid-cache.org/Doc/config/logformat/>

However, you will need to do that for a separate log to other traffic
and as mentioned above keep in mind that port-443 does not necessarily
mean HTTPS.

To actually log https:// URL requires either passing Squid https:// URLs
instead of CONNECT request, or decrypting the traffic (with SSL-Bump
feature) and see what is inside the TLS (if it is TLS, it may not be).
Squid will then log the appropriate https:// URL for each received or
decrypted HTTPS request, no changes necessary.

PS: If you are asking this because of some tool that is doing broken
things when passed real URIs (not URL ... *URI*) that tool needs to be
fixed.

Amos