[squid-users] clarifying Features/SslPeekAndSplice on wiki + fake CONNECT

Tue Aug 23 02:14:56 UTC 2016

Thanks for your reply.
I will start changing the wiki page.
When I think I am done, I will let you know for a review.

What is left is my desire to get a fake CONNECT with FQDN (see below).

Marcus

On 08/22/2016 04:20 PM, Alex Rousskov wrote:
> On 08/21/2016 06:46 AM, Marcus Kool wrote:
>> there are still some issues with the wiki page that I like to clarify.
>
> Thanks a lot for working on this!
>
>
>> wiki: http://wiki.squid-cache.org/Features/SslPeekAndSplice
>>
>> section "processing steps"
>>
>> Can action "none" be removed from step 1?
>
> Done.
>
>
>> Step 1.  what is "CONNECT info"?  In the introduction one speaks
>> of "HTTP CONNECT" which has a FQDN.  But the fake CONNECT sent to
>> a URL rewriter has only an IP address.
>
> CONNECT info is the info associated with a CONNECT request, real or fake
> (depending on the http*_port being used for the transaction). Real
> CONNECT requests often have FQDNs. Fake CONNECT requests have IP
> addresses during step1.
>
>
>> Why is the fake CONNECT that is sent to the URL rewriter done
>> at step 1 where only the IP and not the FQDN is available?
>
> ... because some rewriters want/need those CONNECTs, for various
> reasons. The overall design here is simple and logical in my biased
> opinion: Send everything by default but let admin control what should be
> sent. If _your_ rewriter does not need some CONNECTs, configure Squid
> not to send those unwanted CONNECTS to the rewriter. If Squid lacks ACLs
> to detect unwanted CONNECTs, add support for those missing ACLs.

The fake CONNECT _is_ desired, but with FQDN, to
1) have no differences in the CONNECT information sent to
    the URL rewriter in normal proxy mode and in transparent
    intercept mode.
2) be able to filter.  The url rewriter cannot filter based
    on the IP address, it needs a FQDN/SNI.  The parameter
    %ssl::>sni in url_rewrite_extras is always '-' which is
    not only fatal for _my_ url rewriter.

squid.conf.documented adds to the confusion with
    acl aclname at_step step
    ...
    SslBump1: After getting TCP-level and HTTP CONNECT info.
The forward proxy has a FQDN and the intercept proxy has an IP.
Without this information the confusion is created.

> The url_rewrite_access directive controls what transactions are sent to
> the redirector.
>
> Note that CONNECTs should be sent both during step1 and during step2 by
> default.

I think I missed something.  The URL rewriter on my systems only get IP
addresses, never SNI/FQDN.  And never receives two CONNECTS (i.e. one
at step1 and one at step2).
Can I configure Squid to send a fake CONNECT during step2 ?
What is "during"?  Is the CONNECT sent at the end of step2 so it can
send the SNI?

>> In section "Peek at SNI and Bump" it is stated that SNI is obtained in
>> step 1. This contradicts the text at step 1 and 2.
>
> In section "Peek at SNI and Bump" it is NOT stated that SNI is obtained
> in step 1. It is stated that SNI is obtained by peeking during step 1.
> "obtaining SNI" and "peeking" are not the same! If, during step 1, you
> tell Squid to peek, then, during step 2, Squid peeks and obtains SNI.
>
> There is no contradiction, but we are definitely struggling with how to
> describe what is going on. Specific improvement suggestions are very
> welcomed.
>
>
>> squid.conf.documented has this:
>> #  At each SslBump step, Squid evaluates ssl_bump directives to find
>> #  the *next* bumping action (e.g., peek or splice).
>> Emphasis on "*next* bumping action".
>> Should the wiki be reworded and use "next bumping action"?
>
> Please suggest complete specific fixes. We know that the current wiki
> text is difficult to follow for many so the attempts to rework it are
> welcomed.
>
>
>> Section "Actions"
>>
>> item peek: Receive client SNI (step1), or server certificate (step2)  ...
>> This contradicts the explanation of the processing steps.
>> item stare: likewise.
>
> I do not think it does (see the "peeking during step1 tells Squid to get
> SNI during step2" discussion above), but please suggest a better way to
> phrase those descriptions. We know they are confusing. SNI (and
> certificate parsing) happens at the beginning of stepN+1 but whether
> that step happens at all depends on the stepN action.
>
>
>> I suggest to remove the "Deprecated actions" or use a BOLD warning not
>> to use them.
>
> I do not think we should remove them because comparing their description
> with the currently supported actions may be very useful for upgrading
> admins. Does not Squid already emit a deprecation WARNING when those
> actions are used?
>
> If you know how to gray-out the rows with deprecated actions, please do
> that. Otherwise (or in addition to that), please feel free to add a
> "Avoid this deprecated, poorly supported, and soon to be deleted
> action." or similar text to each deprecated action description.
>
>
>> Section "Mimicking SSL client Hello properties when staring"
>> The section has "The information in this section is incomplete and
>> somewhat stale."
>> Is there any information to update this section ?
>
> Yes, but it is only Squid sources and possibly commit log :-(.
>
>
>> section "Examples"
>>
>> I usually write explicit rules like this:
>
>> # do not touch servers where ssl-bump breaks HSTS
>> acl tls_allowed_hsts ...
>> # prevent bumping some allowed servers with self-signed certificates
>> acl tls_allowed_selfsigned ...
> ...
>
> I recommend against documenting ACLs as actions. Your tls_allowed_hsts
> does not prevent touching anything. Your tls_allowed_selfsigned does not
> prevent bumping of any connections. Etc. ACLs are conditions. They only
> identify certain transactions. They do not touch/prevent/allow/block
> anything. The rules that use those ACLs do all that.
>
>
>> # TLS/SSL bumping steps
>> ssl_bump peek   tls_s1_ip_connect   all                 # peek a client connecting at IP level
>> ssl_bump splice tls_s2_client_hello tls_to_splice       # splice some: do not bump/interfere
>> ssl_bump stare  tls_s2_client_hello all                 # connect to the server and stare(peek) at its TLS/SSL properties
>> ssl_bump bump   tls_s3_server_hello all                 # bump if we can (tls_s2_client_hello/stare succeeded)
>
>
>> The above rules follow the steps of the wiki page but
>> the examples on the wiki have optimised rules and sometimes
>> there are messages on the mailing list that rules can be optimized
>> to save CPU cycles.
>
>
> It is not primarily about CPU optimization. It is about readability.
>
>> The last 4 lines of the example can be optimized into
>
>> ssl_bump peek   tls_s1_ip_connect   all
>> ssl_bump splice tls_s2_client_hello tls_to_splice
>> ssl_bump stare  all
>> ssl_bump bump   all
>
> Actually, they should be written as:
>
>   ssl_bump peek   tls_s1_ip_connect
>   ssl_bump splice tls_to_splice
>   ssl_bump stare  all
>   ssl_bump bump   all
>
>
> And, most likely (possibly after some ACL adjustments) can be further
> clarified as:
>
>   ssl_bump splice tls_to_splice
>   ssl_bump stare  all
>   ssl_bump bump   all
>
> Which is a lot clearer for some, but not for others.
>
>
>> I prefer not to optimize since reading the rules is
>> easier and one understands better what happens at each step.
>
> It is a matter of taste.
>
>
>> However, if a significant number of CPU cycles are saved, I suggest to
>> include an example like above and rules of thumb on how to optimize and the
>> final optimized results so that a less experienced reader understands
>> the optimizations.
>
> Please do, but this is not about optimization. It is about describing
> desired outcome in two different ways:
>
> * If you want to be verbose and prefix every rule with the step it
> applies to, then you can do that (at the elevated risk of forgetting to
> tell Squid what to do in some cases among all the noise).
>
> * If you want to express the essence of your configuration, you can do
> that instead (at the risk of misinterpreting which step a rule will be
> applied at, due to increased complexity).
>
>
> There are only two "noise reduction" or "condensing" rules AFAICT:
>
> 1. Do not say "large green all". Just say "large green".
> 2. Splice and peek rules are ignored during step 3.
>
>
>
>> The examples in "Avoid bumping banking traffic" :
>> I can image what used (but not defined) serverIsBank acl looks like:
>>    acl serverIsBank  ssl::server_name .santander.com
>> but have no idea what the acl haveServerName looks like.
>> What is intended here?
>
> IIRC, that example was originally written before ssl::server_name
> existed. haveServerName would have to detect a request where the server
> name is known (e.g., step2 with SNI or step3). The difficulties with
> writing such ACLs is what prompted us to add ssl::server_name. The
> example should probably be rewritten as:
>
>   acl serverIsBank ssl::server_name ...
>   ssl_bump peek step1
>   ssl_bump splice serverIsBank
>   ssl_bump bump all
>
> or even
>
>   acl serverIsBank ssl::server_name ...
>   ssl_bump splice serverIsBank
>   ssl_bump stare all
>   ssl_bump bump all
>
> if you do not want Squid to silently bump bank transactions that lack
> SNI. I am not sure whether Squid will log level-1 warning when it cannot
> splice after staring during step2, but it would be appropriate to teach
> Squid to do so IMO (with a rate/count limit of course).
>
> The example description would need to be adjusted accordingly.
>
>
> Thank you,
>
> Alex.
>
>
>