[squid-dev] [RFC] simplifying ssl_bump complexity

[Marcus Kool]

On 11/28/2016 03:58 PM, Alex Rousskov wrote:
> On 11/28/2016 06:30 AM, Marcus Kool wrote:
>> On 11/27/2016 11:20 PM, Alex Rousskov wrote:
>>> It would be nice to prohibit truly impossible actions at the syntax
>>> level, but I suspect that the only way to make that possible is to focus
>>> on final actions [instead of steps] and require at *most* one ssl_bump
>>> rule for each of the supported final actions:
>>>
>>>   ssl_bump splice    ...rules that define when to splice...
>>>   ssl_bump bump      ...rules that define when to bump...
>>>   ssl_bump terminate ...rules that define when to terminate...
>>>   # no other ssl_bump lines allowed!
>>>
>>> The current intermediate actions (peek and stare) would have to go into
>>> the ACLs. There will be no ssl_bump rules for them at all. In other
>>> words, the admin would be required to _always_ write an equivalent of
>>>
>>>   if (a1() && a2() && ...)
>>>   then
>>>       splice
>>>   elsif (b1() && b2() && ...)
>>>   then
>>>       bump
>>>   elsif (c1() && c2() && ...)
>>>   then
>>>       terminate
>>>   else
>>>       splice or bump, depending on state
>>>       (or some other default; this decision is secondary)
>>>   endif
>>>
>>> where a1(), b2(), and other functions/ACLs may peek or stare as needed
>>> to get the required information.
>
>> The above if-then-else tree is clear.
>
> It is clear at the top level. I am not sure we can also make the
> secondary level (i.e., intermediate peak/state actions) clear because
> they have to be side effects in this design and side effects are rarely
> clear. We can try though.
>
>
>> I like your suggestion to drop steps in the configuration and make
>> Squid more intelligent to take decisions at the appropriate
>> moments (steps).
>
> Please note that the current configuration does not contain implicit
> steps either; in its spirit/intent, it is essentially the same as the
> above three-action sketch. Needless to say, most correct configurations
> do contain an explicit step ACL or two, but they are supposed to be used
> to essentially implement the above three-action logic.
>
> Also, to avoid misunderstanding, I am not (yet) advocating any specific
> configuration approach, including the one I started to sketch above. I
> am only documenting a possible solution to the "How to make impossible
> actions invalid at the syntax level" problem you have raised.

Yeah, I used a bit optimistic wording.  It cannot be solved at the syntax
level, but should be possible at the semantic level.

>> You mentioned admins being surprised about Squid bumping for a
>> notification of an error and one way to improve that is to replace
>> 'terminate' by 'terminate_with_error' (with bumping) and 'quick_terminate'
>> (no bumping, just close fd).  The quick_terminate, if used, is also
>> faster, which is an added benefit.

I replied to the item where admins get confused when Squid bumps
to generate an error and suggested a way to control the behavior
of Squid to terminate instead of bump and generate an error.

> Terminate is always an instant TCP connection closure, without any
> bumping and without any errors being delivered to the user.
>
> AFAIK, admins are not surprised by bumping when a terminate rule matches
> (that would be a Squid bug). Admins are surprised by bumping when a
> splice, peek, or stare rule matches. That surprise is a different
> problem (that we should also try to solve, of course -- see the three
> "core problem" bullets in my original response).


>
>> The only comment that I want to make without starting a
>> new thread is that I think that conceptual terms are better
>> than technical terms (hence my preference for 'passthrough'
>> instead of 'splice').  But let's save this discussion for later.
>
> Personally, I doubt we should spend much time discussing whether
> "passthrough" is better than "splice", but I agree that nobody should be
> discussing naming until the much bigger (and, hopefully, solvable)
> problems are solved.
>
>
>>> I suspect that not looking at some SSL Hellos will always be needed
>>> because some of those Hellos are not Hellos at all and it takes too much
>>> time/resources for the SSL Hellos parser to detect some non-SSL Hellos.
>>> Besides that, it is always nice to be able to selectively bypass the
>>> complex Hello parser code in emergencies.
>
>
>> Perhaps less resources are used if there is a two-stage parser:
>> 1) quick scan of input for data layout of a ClientHello
>> 2) do the complex parsing.
>
> Squid v4 already uses a two-stage Hello parser (the first stage is built
> into Squid and the second stage is provided by OpenSSL).
>
> You are oversimplifying the general protocol recognition problem because
> you are focusing of the trivial cases. For example, one of the reasons
> the first-stage parser cannot be always "quick" is because, in some
> cases, there is no data to parse -- the server has to speak before the
> client will send anything. Another example is SSL-like traffic that is
> not really SSL. And a third example is Squid/OpenSSL limitations in
> handling advanced valid SSL traffic. In summary, there will always be a
> need for bypass IMO.

I am not suggesting to drop a bypass.
The comment was about the fact that determination if data from a client
is a syntactically valid ClientHello message should be quick and without
use of OpenSSL functions.  Since I do not know all SSL-like traffic of
all applications it is open for debate if it is possible, but I remain
optimistic that it can be done.

When a server sends data first, it is not a TLS handshake.
So when a server sends data first, Squid knows quickly (if the server
is quick) that the connection is not a TLS connection, right?

>>>>> 3) the "TLS server hello" step:
>
>>> all proposals have to decide how to distinguish stare from peek
>>> during step2. Those two intermediate actions result in different bytes
>>> on the wire so they cannot be mapped to a single verb like "continue".
>
>> An issue that complicates things is an "impossible decision",
>> the desire to splice when "it is too late".  I suggest to
>> solve this issue by simply splicing a bumped connection.
>
> We can and probably should teach Squid to bump and tunnel stared at
> connections as needed (I bet this can be done within the current
> configuration paradigm). However, this does not help with distinguishing
> two valid "continue" actions (peek at the server vs. stare at the
> server). The admin has to tell Squid what to do (either peek or stare).
> A single "continue" action does not/cannot carry that information.

It can help.  Suppose one want to make a decision based on the server
certificate, then currently one must use stare at step2 and currently
this has a limitation that it cannot be spliced at step3.
With a splice-bumped-connections feature this issue is resolved by
staring at step2 and make a splice-or-bump decision at step3.
So the decision that currently is enforced at step2 can be postponed
by doing a 'continue' and decide at step3.

>>>>> The configuration directives as I proposed are IMO intuitive and
>>>>> leave very little room for misunderstandings.
>>>
>>> ... if you use them correctly. However, the same is true for the current
>>> directives so this is not really an illustration of improvement AFAICT.
>>> For example, what does the following [mis]configuration do?
>>>
>>>   https_decryption off
>>>   tls_server_hello continue
>>>   tls_server_hello terminate hacked_server
>>>   https_decryption on
>>>   tls_client_hello continue
>>>   tls_client_hello passthrough banks
>
>> You are good in finding examples of nonsense configurations :-)
>
> I am just thinking from a confused user point of view. All the proposals
> I have seen so far do not help much (and may hurt) when the admin
> already knows what she is doing. To justify a configuration change, an
> example has to illustrate an important _difference_ for a [confused]
> user rather than just illustrate the new configuration.
>
>
>> the config can just have a list
>> of rules and instead of Squid obeying the order of the TLS rules at
>> all times, it can gain intelligence and at the time that Squid is done
>> reading the configuration, validates whatever it can validate and makes
>> the decisions about which acls are executed/verified at what time (and
>> also does some runtime checks at the appropriate times).
>
> This "intelligence gathering" cannot gather much information because
> Squid cannot evaluate most ACLs outside a specific transaction context.
>
> Also, it might be important to emphasize the primary point of ordered
> rules (in general): Squid and lots of other software use ordered rules
> not because they want to force the admin to think in some specific
> "steps" order. They use ordered rules to implement the "else" logic. If
> rules are unordered, then to implement "if A then splice else bump"
> logic, one has to write "If A then splice. If !A then bump". This
> duplication of ACLs leads to configuration bugs and performance waste
> (among other bad things).
>
> This order support is why I wrote "elsif" in my earlier three-action
> sketch instead of just writing "if" three times.

Order is often important, but Squid can gain more
intelligence by upfront making decisions about which rules
get evaluated when.  For example, 'splice banks' can only be
done when the server_name is known so after a peek and can be skipped
when the server_name is not yet known.

>> So the config for 'bump all except banks' could look like this:
>> tls decryption on
>> tls default bump
>> tls splice banks
>> tls terminate clients_from_subnet_x
>> tls terminate hacked_server
>> tls splice no_http_protocol_inside_tls_wrapper  # default
>> tls splice no_tls_protocol                      # default
>> tls terminate tls_without_sni
>>
>> The above is elegant and I think that the required intelligence to
>> decide to evaluate which rule at what step is not impossible.
>
> This is already 90% supported AFAICT:
>
> + Squid already skips ssl_bump directives with currently impossible
> actions and does not require explicit steps. The above config can be
> written almost as is using the current configuration approach and almost
> everything will "work" as intended (see the two bullets below for
> exceptions).
>
> - One invisible part of your example is not supported: There is
> currently no way for an ACL to trigger a peeking or staring action to
> gather more information. Whether it is a good idea to add ACLs with such
> side effects is an open question. *If* they are added, the above
> configuration will work with just one last minor change:
>
> - The "tls default bump" has to be listed as the last "tls bump all"
> rule. Adding a special "default" rule is also possible, of course, but I
> doubt that it is a good idea because admins are already used to
> ACL-driven rules to end with the default action.
>
> Your own "splice ... # default" rules that contradict the "default bump"
> setting is also a good illustration why a dedicated default directive
> may not be a good idea -- there are often many "defaults" because Squid
> "default" action often depends on the current transaction context.

If you interpret 'tls default bump' as 'the objective is to bump'
and 'splice banks' as an exception to the default behavior, they do
not contradict.

IMO your 3-bullet list misses these:
* improve the syntax/rules since currently it confuses too many admins
* make the configuration simpler where an admin does not have
   to know many details of the TLS protocol.

The first bullet is why Amos and I tried to improve the
situation by syntactical changes.

The last bullet may also be interpreted as: make the config
rules simple for 95% of all sites and support optional complex rules
for the rest.

>> Squid must not allow a mix of 'tls decryption on' and
>> 'tls decryption off'.
>
> Mixtures can be prohibited but such prohibitions often create problems
> for [partially-generated] configurations. IMO, a better approach is to
> infer whether "decryption" (whatever it is) is needed based on the
> actual rules. This is what the current implementation does.
>
>
>> Simple examples like above 'work'.  The question is if it
>> is possible for Squid to detect most or all nonsense configurations
>
> I know the answer to that question: "No". Complex actions like bumping
> traffic require human intelligence. Squid will not be able to detect
> bugs in advanced human thinking (in the foreseeable future). You cannot
> make complex bumping both highly configurable and fool-proof. If you
> want a fool-proof solution, I know of three possible ways:
>
> * Reduction: Drastically reduce the number of supported configurations
> and uses. Support just a few simple fool-proof cases. I think this
> solution is unacceptable for a product like Squid and for a complex area
> such as SSL bumping.

An alternative for Reduction is a list of good examples (better than
wiki has).

> * Wizard: Provide an intelligent configuration generator/wizard which
> asks admins some questions and then generates a valid squid.conf snippet
> while guiding and educating the admin. A wizard may also evaluate a
> given configuration, with human assistance (a much harder but not
> impossible task). No Squid changes are necessary.

Would like to see that :-)

> * Shim: Provide a set of simple new directives that cover basic bumping
> needs. Internally, these directives are mapped to the already supported
> "power-user" configuration. Admins using the new directives will not be
> allowed to add old/power-user directives into the mix (i.e., the two
> sets of directives will be mutually exclusive). This is a good approach
> *if* we can come up with new fool-proof directives that cover many
> bumping needs well.

Shim is nice, but does not change the current situation for power-users.

> Currently, this thread appears to be focused on another alternative:
> Replacing the current set of directives with a much simpler one while
> supporting the same functionality. This is an honorable goal, but I do
> not think you can sneak "detection of most nonsense configurations" into
> that. If the latter is your goal, see the three items above.

Going back to your 3 bullets,
* Can you give an example of more dynamic ACLs?
   I am surprised that admins are surprised that Squid does server
   certificate validation.  Maybe document it better?
* the issue with price becomes less if splice-a-bumped-connection
   is implemented and also solves the problem of precluding future
   decisions.
* I suggest to document that Squid bumps a connection to send
   an error message.  Maybe error handling can be separated from
   the other ssl bump rules and have a list of acls on how to act
   (terminate or bump+error) in case there is an error.

Marcus

>
> HTH,
>
> Alex.