[squid-users] server persistent connections and cache

[Alex Rousskov]

[There is a potentially useful reframing of the question at the end if
you want to skip the details...]

On 07/26/2018 11:11 PM, Amos Jeffries wrote:
> On 27/07/18 16:18, Alex Rousskov wrote:

>> one could argue that Squid should honor a
>> (higher level) client and server assumption that they are talking to
>> each other (at HTTP+ level).

> Configuring persistence on/off is the way to control that, not pinning.

I disagree. Persistence is about a single connection. Pinning is about
the relationship between two connections. Honoring tunneling
expectations can be about pinning, but it is not about persistence.
Naturally, there is no pinning without persistence, but that is
irrelevant for designing how to support tunneling expectations.


>> We will probably break fewer transactions
>> that way. With that idea in mind, the "first point" becomes establishing
>> a TCP tunnel with the server, even if no client Hello pieces are forwarded.

> HTTP(S) being stateless, at that level any endpoint relying on implicit
> state is non-compliant and buggy.

What is known as HTTP statelessness is irrelevant here: HTTP CONNECT
tunnel itself is a "state"; agents have the right to rely on that state.
That state is not a state that relates to HTTP messages so there is no
contradiction with HTTP stateless principles. From HTTP point of view,
there are no HTTP messages inside that tunnel. The tunnel is a monolith
at HTTP level. Not treating it as such will break some HTTP-compliant
agents.

Same for intercepted TLS/TCP connections -- the preservation of those
TLS and TCP connections is something an HTTP-compliant agent can
legitimately rely on.


> It is *nice* not to result in visible errors, but not a requirement we
> should stick to at cost of proper behaviour.

The open question is how to define what is "proper". Any behavior will
have pros and cons associated with it. You cannot declare lack of
pinning a "proper behavior" just because most agents do not require
pinning. The decision is more complex than that because there are more
variables involved.


> The client accepted proof that Squid *was* that origin (false or not)
> order to reach said higher levels.

I disagree that being content with Squid certificate (TLS level)
constitutes acceptance of an intermediary that can switch from one
server to another at will (higher level).


> Pinning at a later time due to higher
> level stateful situation is not relevant to the bumping code actions and
> not something we need to consider at the present C2 use of S after C1
> should have pinned it.

Agreed. I do not think that contradicts what I was (and am) saying. The
question here is whether "C1 should have pinned S" after bumping C1 at
step1.


> "after C1 has pinned" - if no pinning happens at all the whole
> statement is irrelevant.

Correct. The question is whether C1 should pin S after bumping at step1.
Today, no such pinning happens, but it could be argued that this is a
bug. It is a complicated question with many variables and no obvious
answers.


> What I'm most confused about here is why S is closed when C1 dies with
> non-pinning.  _unless_ there is pinning between them they should not be
> that closely fate sharing.
> 
> Is the HTTPS message on both C1 and S saying "Connection: close" perhapse?

or perhaps some kind of still-linked FwdState/Client abort if C1 was not
closed properly/orderly.



> Off-topic; TLS has connections that do not use handshakes at all, which
> are becoming more common in TLS/1.3. So I believe such a feature may be
> coming one day, but irrelevant right now.

IIRC, all TLS versions, including v1.3, start a TLS connection with a
handshake. The scope/meaning of that handshake varies across versions,
but there is always some handshake.


>> The open question is whether S should be pinned in the case where C1 is
>> bumped at step1 (i.e., "Case 1" in Vishali's email).

> IMO in *that* specific case S should not be pinned. Because pinning
> would prevent the very reasonable actions of handling server failures by
> opening new Sn connections to finish incomplete responses to the client,
> or retaining a server connection for the quick_abort features.

IMO in that specific case S should be pinned by default because pinning
will break fewer transactions and any breakage would be easier to
triage/explain. That default may also be more "secure" because without
pinning, C1 can start requesting content from other servers, possibly
violating admin expectations that rely on vetting C1 based on TLS-level
information such as SNI).

What do we do now for intercepted HTTP clients -- are they allowed to
request content from a server they were _not_ going to when intercepted?



> Squid should instead be ensuring that the server handshake at TLS level
> matches what it would have used for C2 clean handshake. I may have
> missed it, but that is not yet being done (just TCP equivalence, not TLS).

If we do not honor the tunneling semantics (i.e., do not pin S1), then I
see no reason to ensure that S1 and S2 handshakes match. S2 can even go
to a different server.


The same question can be formulated in a more general way, which may be
useful when comparing what Squid already does across different
configurations/scenarios: Which of the following should be "pinned"?

0. Nothing (free to switch to a different origin server).
1. Destination server name (free to switch to a different IP).
2. Destination IP:port (free to reestablish or reuse a connection).
3. Destination connection (free to serve cache hits).
4. Destination (no freedom at all; like regular tunnels today).

For Case1, Squid probably uses #0 or #1 pinning today. We need to decide
whether Squid should use #2 or #3 instead.

For Case2, Squid uses #3 pinning today AFAICT.

#4 is probably best addressed using the serve_hit directive.


Alex.