[squid-users] acl / format code evaluation

[Alex Rousskov]

On 11/4/21 11:14 AM, Jason Spashett wrote:

> I am using squid 5, and after reading the following I have attempted
> to link the connect requests to the other requests within a TLS
> tunnel.
> http://lists.squid-cache.org/pipermail/squid-users/2021-April/023526.html

> I added an extra log format code to squid 5, called %random, which
> always returns a random string, in the hopes of using this to stamp
> against log entries to tie them together.

It is difficult for me to imagine the right use case for identifying
clients with random strings, especially if you are adding new %codes:

* If you want uniqueness within a Squid kid, then you should use a
simple ever-increasing counter, not a random string.

* If you want uniqueness across kids, processes, and instances, then you
probably need a UUID, not just a random string.


> The squid configuration follows, this seems to partially work in that
> the requests in the tunnel have the same conn_id, but the connect
> request itself has a different one, which leads me to to believe that
> the format code is being evaluated twice in that case perhaps.

Bugs notwithstanding, Squid applies the note directive once for every
client transaction. That includes one or more CONNECT transactions and
each GET/POST/etc. transaction inside the bumped CONNECT tunnel. Squid
applies the note directive at the end of the client transaction
lifetime. The resulting annotations may be logged (via %note) when/if
the transaction is logged.


> Note that I added %master_xaction to the log too, to see if that
> worked, and it does, but it's not particularly practical on it's own
> due to the problem of it not being unique enough.

Whether it is unique enough depends on the desired uniqueness scope. I
bet that, strictly speaking, it is no less unique than your current
%random algorithm!

The first CONNECT request on a TCP connection should have a unique
master transaction ID compared to all other CONNECT requests on other
TCP connections accepted by the same kid process. That is exactly what
you need if you need per-process uniqueness. If you need a larger
uniqueness scope, then you probably need to use UUIDs.


> Can anyone tell me why this isn't working,

I am not sure at all, but it could be a Squid bug related to how client
annotations are propagated across CONNECT and in-tunnel requests. I am
also worried about the note directive application time, but I cannot
offer a specific explanation on how it can break things in your use case.


I suggest that you revise your configuration to:

1. Avoid the note directive. It might be applied too late, and it is not
meant to do what you are trying to accomplish.

2. Use ssl_bump directive to annotate the client during SslBump step1.
This directive is evaluated early, even before http_access. For plain
requests, you can use http_access.

3. Continue to annotate each client exactly once. Replace "+=" with "="
to avoid creating the impression that you are updating annotations.


> and or when the log format codes get evaluated.

They are evaluated when used. In your specific case, you have not told
us how your configuration uses logformat "log", but if you are using
"log" just for the access_log directive, then "log" %codes are evaluated
when Squid creates the access.log record for each transaction.


HTH,

Alex.



> 
> Squid configuration
> -------------------
> #
> acl connection_id_acl annotate_client conn_id+="%master_xaction/%random"
> acl has_conn_id_acl note conn_id
> acl set_conn_id_once_acl any-of has_conn_id_acl connection_id_acl
> note "" "" set_conn_id_once_acl
> #
> logformat log time="%tl" conn_id=%{conn_id}note request_type=%>rm url=%>ru
> 
> log output
> ----------
> time="04/Nov/2021:14:54:19 +0000" conn_id=2550/Fh0Lje1 request_type=CONNECT url=blog.jason.spashett.com:443
> time="04/Nov/2021:14:54:19 +0000" conn_id=2550/e5sVhqi request_type=GET url=https://blog.jason.spashett.com/minecraft-4k-ported-to-the-d-programming-language/
> time="04/Nov/2021:14:54:20 +0000" conn_id=2550/e5sVhqi request_type=GET url=https://blog.jason.spashett.com/css/main.css
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>