[squid-dev] [RFC] simplifying ssl_bump complexity

[Alex Rousskov]

On 11/28/2016 03:26 PM, Marcus Kool wrote:

> The comment was about the fact that determination if data from a client
> is a syntactically valid ClientHello message should be quick

Unfortunately, it is not a fact. In simple cases, you are right (which
is why Squid v4 already uses a two-stage parser). In other cases, it may
cost time or even require taking some risks to make the "client speaks
TLS" determination. This is why bypass support is required _before_ any
TLS parsing starts.


> When a server sends data first, it is not a TLS handshake.

I am not sure that is 100% accurate because there could be TLS servers
that send the first byte(s) early (for some stupid reason), but even if
we are sure that such servers do not exist, Squid cannot know whether
the server is going to speak first unless Squid has _connected_ to the
server already (and waited for some time). In some cases, the admin does
not want Squid to contact the server (even at TCP level) until Squid is
sure that the connection is carrying TLS with validated SNI.


> So when a server sends data first, Squid knows quickly (if the server
> is quick) that the connection is not a TLS connection, right?

No quite, unfortunately. See my previous message and the above example
for the cases where nothing is quick or simple at the expected-TLS level.



>>>>>> 3) the "TLS server hello" step:
>>
>>>> all proposals have to decide how to distinguish stare from peek
>>>> during step2. Those two intermediate actions result in different bytes
>>>> on the wire so they cannot be mapped to a single verb like "continue".
>>
>>> An issue that complicates things is an "impossible decision",
>>> the desire to splice when "it is too late".  I suggest to
>>> solve this issue by simply splicing a bumped connection.
>>
>> We can and probably should teach Squid to bump and tunnel stared at
>> connections as needed (I bet this can be done within the current
>> configuration paradigm). However, this does not help with distinguishing
>> two valid "continue" actions (peek at the server vs. stare at the
>> server). The admin has to tell Squid what to do (either peek or stare).
>> A single "continue" action does not/cannot carry that information.

> Suppose one want to make a decision based on the server
> certificate, then currently one must use stare at step2 and currently
> this has a limitation that it cannot be spliced at step3.
> With a splice-bumped-connections feature this issue is resolved by
> staring at step2 and make a splice-or-bump decision at step3.

Yes, tunneling of bumped connections is useful in _some_ cases and
should be supported, but it does not eliminate the a peek-vs-stare
decision at step2. If you assume that it does eliminate that decision,
then you have to assume that bumping always works, which is obviously
not true.

* If Squid is talking to a server that cannot be successfully bumped,
the [future] support for tunneling of bumped connections does not help,
and the admin has to tell Squid to peek rather than stare.

* If Squid is talking to a server that can and should be successfully
bumped, the [future] support for tunneling of bumped connections helps,
but the admin has to tell Squid to stare rather than peek.

Thus, a single "continue" action does not work -- it will not
accommodate one of the two bullets above.

[ FWIW, I strongly discourage you from applying the term "splice" to
bumped connections -- it is easy to deceive oneself that staring is not
needed because all connections end up "spliced". I recommend saying
"tunneled" when talking about bumped connections passed through Squid
(as opposed to spliced connections passed through Squid). ]


> So the decision that currently is enforced at step2 can be postponed
> by doing a 'continue' and decide at step3.

AFAICT, your logic is missing the fact that, if step2 is not final, then
Squid has to do one of two actions during step2: stare or peek. Those
are different actions, resulting in different bytes on the wire. You
cannot postpone that peek-vs-stare decision until the final step3. Thus,
one "continue" action does not work.

Again, the requirement for two "continue" actions does not imply that
tunneling of bumped connections is not helpful in some cases!


> Squid can gain more
> intelligence by upfront making decisions about which rules
> get evaluated when.  For example, 'splice banks' can only be
> done when the server_name is known so after a peek and can be skipped
> when the server_name is not yet known.

That example does not compute for me: To know whether the server_name is
known, Squid has to evaluate the server_name ACL. If that evaluation is
possible, Squid should just use its result (and skip or honor the
"splice banks" rule) instead of speculating that the rule can be safely
skipped during step1.

In real configurations, most rules cannot be correctly evaluated outside
transaction context because they contain ACLs, such as the
ssl::server_name ACL, that are context-dependent.


>>> So the config for 'bump all except banks' could look like this:
>>> tls decryption on
>>> tls default bump
>>> tls splice banks
>>> tls terminate clients_from_subnet_x
>>> tls terminate hacked_server
>>> tls splice no_http_protocol_inside_tls_wrapper  # default
>>> tls splice no_tls_protocol                      # default
>>> tls terminate tls_without_sni

>> Your own "splice ... # default" rules that contradict the "default bump"
>> setting is also a good illustration why a dedicated default directive
>> may not be a good idea -- there are often many "defaults" because Squid
>> "default" action often depends on the current transaction context.

> If you interpret 'tls default bump' as 'the objective is to bump'
> and 'splice banks' as an exception to the default behavior, they do
> not contradict.

I do not think many reasonable people can interpret "... # default" as
"an exception to the default behavior" but YMMV.


>>> The question is if it
>>> is possible for Squid to detect most or all nonsense configurations

>> I know the answer to that question: "No". Complex actions like bumping
>> traffic require human intelligence. Squid will not be able to detect
>> bugs in advanced human thinking (in the foreseeable future). You cannot
>> make complex bumping both highly configurable and fool-proof. If you
>> want a fool-proof solution, I know of three possible ways:
>> 
>> * Reduction: Drastically reduce the number of supported configurations
>> and uses. Support just a few simple fool-proof cases. I think this
>> solution is unacceptable for a product like Squid and for a complex area
>> such as SSL bumping.
>> 
>> * Wizard: Provide an intelligent configuration generator/wizard which
>> asks admins some questions and then generates a valid squid.conf snippet
>> while guiding and educating the admin. A wizard may also evaluate a
>> given configuration, with human assistance (a much harder but not
>> impossible task). No Squid changes are necessary.
>> 
>> * Shim: Provide a set of simple new directives that cover basic bumping
>> needs. Internally, these directives are mapped to the already supported
>> "power-user" configuration. Admins using the new directives will not be
>> allowed to add old/power-user directives into the mix (i.e., the two
>> sets of directives will be mutually exclusive). This is a good approach
>> *if* we can come up with new fool-proof directives that cover many
>> bumping needs well.
>> 
>> 
>> Currently, this thread appears to be focused on another alternative:
>> Replacing the current set of directives with a much simpler one while
>> supporting the same functionality. This is an honorable goal, but I do
>> not think you can sneak "detection of most nonsense configurations" into
>> that. If the latter is your goal, see the three items above.


> IMO your 3-bullet list misses these:
> * improve the syntax/rules since currently it confuses too many admins
> * make the configuration simpler where an admin does not have
>   to know many details of the TLS protocol.

Please do not forget that the Reduction/Wizard/Shim bullets only apply
to your "detect most or all nonsense configurations" goal. What you are
probably implying above is covered by the "Replacing the current set of
directives with a much simpler one while supporting the same
functionality" paragraph after those bullets.


> The first bullet is why Amos and I tried to improve the
> situation by syntactical changes.

Not sure what you mean, but if "Reduction" (the first bullet) is your
preferred approach, then we are stuck because I find that approach
unacceptable for obvious (IMO) reasons.


> The last bullet may also be interpreted as: make the config
> rules simple for 95% of all sites and support optional complex rules
> for the rest.

I would not recommend that way of thinking/phrasing because it may imply
that an average admin will use a _mixture_ of directives from simple and
complex sets. If that is the result, you have accomplished nothing
because the current problems will continue to exist for that average
admin using complex directives (that he should avoid).

For the "Shim" approach to work, the set of simple directives should
satisfy 100% of an average admin needs so that an average admin does not
have to switch to (or, worse, mix in!) directives from the complex set.
A significant barrier should prevent an average admin from considering
using complex rules. The requirement to use complex rules for either
nothing or everything could be such a barrier.


>>> Simple examples like above 'work'.  The question is if it
>>> is possible for Squid to detect most or all nonsense configurations

>> I know the answer to that question: "No". Complex actions like bumping
>> traffic require human intelligence. Squid will not be able to detect
>> bugs in advanced human thinking (in the foreseeable future). You cannot
>> make complex bumping both highly configurable and fool-proof. If you
>> want a fool-proof solution, I know of three possible ways:
>>
>> * Reduction: Drastically reduce the number of supported configurations
>> and uses. Support just a few simple fool-proof cases. I think this
>> solution is unacceptable for a product like Squid and for a complex area
>> such as SSL bumping.

> An alternative for Reduction is a list of good examples (better than
> wiki has).

You may have missed the context of this Reduction bullet: A list of good
examples does not allow Squid to detect most or all nonsense configurations.


>> * Wizard: Provide an intelligent configuration generator/wizard which
>> asks admins some questions and then generates a valid squid.conf snippet
>> while guiding and educating the admin. A wizard may also evaluate a
>> given configuration, with human assistance (a much harder but not
>> impossible task). No Squid changes are necessary.

> Would like to see that :-)

Me too! FWIW, we have developed a somewhat similar configuration wizard
for Web Polygraph, and I am pretty sure it is possible to create one for
SslBump. The standard FAQ applies even though this project may not
require any Squid modifications:
http://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F


>> * Shim: Provide a set of simple new directives that cover basic bumping
>> needs. Internally, these directives are mapped to the already supported
>> "power-user" configuration. Admins using the new directives will not be
>> allowed to add old/power-user directives into the mix (i.e., the two
>> sets of directives will be mutually exclusive). This is a good approach
>> *if* we can come up with new fool-proof directives that cover many
>> bumping needs well.

> Shim is nice, but does not change the current situation for power-users.

True (but missing the context of these bullets again).


>> Currently, this thread appears to be focused on another alternative:
>> Replacing the current set of directives with a much simpler one while
>> supporting the same functionality. This is an honorable goal, but I do
>> not think you can sneak "detection of most nonsense configurations" into
>> that. If the latter is your goal, see the three items above.
> 
> Going back to your 3 bullets,

Your questions do not seem to be related to the bullets. I will answer
them in a general context:


> * Can you give an example of more dynamic ACLs?

For example, we can add a get_client_hello and get_server_hello ACLs
that will trigger a peeking or staring actions as necessary. They will
either always match or will match if the action was successful. The
get_server_hello ACL will need options to distinguish peeking from staring.

  acl got_client_hello get_client_hello
  acl client_goes_to_bank server_name ...

  ssl_bump splice got_client_hello client_goes_to_bank
  ssl_bump bump all

Needless to say, peeking/staring ACLs do not solve the "too late to
splice!" problem. For example, neither of the following examples
magically start to work:

  # XXX: One cannot tunnel after staring at the server:
  acl got_server_hello get_server_hello --stare
  acl server_uses_pinning server_name ...
  ssl_bump tunnel got_server_hello server_uses_pinning # XXX
  ssl_bump bump all

or

  # XXX: One cannot bump after peeking at the server:
  acl got_server_hello get_server_hello --peek
  acl server_uses_pinning server_name ...
  ssl_bump splice got_server_hello server_uses_pinning
  ssl_bump bump all # XXX


>   I am surprised that admins are surprised that Squid does server
>   certificate validation.  Maybe document it better?

Better documentation would not hurt but I do not know whether it will
help. Please note that it is not just validation that some admins find
surprising, it is bumping that follows unsuccessful validation. For
example, many think that this configuration will never bump, but IIRC,
it may if validation fails:

  ssl_bump peek all
  ssl_bump splice all


> * the issue with price becomes less if splice-a-bumped-connection
>   is implemented

Yes, but ...

>   and also solves the problem of precluding future decisions.

It does not fully solve that problem as discussed above (one "continue"
does not work).


> * I suggest to document that Squid bumps a connection to send
>   an error message.  Maybe error handling can be separated from
>   the other ssl bump rules and have a list of acls on how to act
>   (terminate or bump+error) in case there is an error.

The error handling is separated (took another fight or two IIRC).
However, there is currently no [convenient] way to terminate the
connection on errors. I agree that such a way should be added, but it is
tricky to do that well because Squid already has a similar TCP_RESET
knob tied to deny_info. A good solution would generalize that existing
functionality instead of creating a parallel control structure with
overlapping/conflicting scope.

Alex.