[squid-users] Reverse proxying Exchange OWA wembail with SSL offloading

[Scott]

On Sat, Oct 31, 2020 at 12:49:16AM +1300, Amos Jeffries wrote:
> On 30/10/20 3:27 pm, Scott wrote:
> > On Thu, Oct 29, 2020 at 10:08:42PM +1300, Amos Jeffries wrote:
> >> On 29/10/20 12:06 pm, Scott wrote:
> >>> On Wed, Oct 28, 2020 at 12:00:01PM +0000, squid-users-reques wrote:
> >>>> Date: Thu, 29 Oct 2020 00:08:34 +1300
> >>>> From: Amos Jeffries
> >>>>
> >>>> On 28/10/20 5:25 pm, Scott wrote:
> >>>>>
> >>>>> Here are the logs (first not working, followed by working).
> >>>>>
> >>>>> Note this is the login attempt, not the loading of the initial page.  You'll
> >>>>> see in the NOT WORKING section that the browser does NOT return a cookie to
> >>>>> the server, which is where the problem may be.  Again, I'm not sure why - I'm
> >>>>> thinking perhaps the browser/javascript is rejecting the cookie as it's
> >>>>> missing the "secure" attribute (because the back-end is talking plain HTTP).
> >>>>>
> >>>>
> >>>> The complete absence of a cookie may be expected to break something.
> >>>>
> >>>> The absence of a "secure" flag should only make the cookie vulnerable to
> >>>> leaking. It should not affect anything depending on that cookies value.
> >>>>
> >>>>
> >>>> Amos
> >>>>
> >>>
> >>> After some more research and experimentation I've confirmed that my
> >>> suspicions are correct.
> >>>
> >>> Recent browsers are no longer accepting cookies with the SameSite flag set
> >>> without the Secure flag set.
> >>>
> >>> It's not an issue with Squid (although one Squid could solve - but I'm unsure
> >>> of the implications).
> >>
> >> Implications are that the server may have intentionally used the
> >> combination it did, no mistakes.
> >>
> >> The server is given "Front-End-Https: On" so that it is aware the client
> >> is using HTTPS and can set (or not) the secure flag appropriately to
> >> what it needs. Squid is not aware of whether the cookie is safe to use
> >> on HTTP or restrict to just HTTPS.
> >>
> >>
> >>>
> >>> Here is a useful link:
> >>> https://docs.microsoft.com/en-us/office365/troubleshoot/miscellaneous/chrome-behavior-affects-applications
> >>>
> >>> I tested Chrome 85 on Windows - the default settings DO NOT allow for these
> >>> cookies.  However after setting
> >>> 	Cookies without SameSite must be secure
> >>> to Disabled, these cookies are permitted and OWA works.
> >>>
> >>> There are obvious implications for sites doing SSL offloading here.  Are
> >>> sites no longer doing SSL offload?  Or are reverse proxies adding the Secure
> >>> flag?
> >>
> >>
> >> Neither. When a site frontend is entirely https:// with no http://
> >> resources mixed in the Secure flag can be used by the server regardless
> >> of what the internal connections are.
> >>
> >>
> >> Amos
> > 
> > My point is that, assuming browsers are now enforcing SameSite cookies must
> > be secure, then doing SSL offload (whereby the origin server does NOT flag
> > the cookie as secure) will break.
> 
> But offloading does not mean the server omits the secure flag.
> 
> Servers which choose to send it when offloading work fine. Servers that 
> choose to omit it have problems with the Google paranoid interpretation 
> of security.

Aren't origin servers oblivious to SSL offloading?  I thought they just 
happily accepted HTTP connections with no knowledge of any secure channel 
between client and reverse proxy.

If that's correct then, although not proscribed by the RFC, it's unlikely 
that a Set-Cookie header would ever contain a Secure flag because the server 
would be saying "I insist this Cookie be transmitted securely, but here it is 
over an insecure channel".

> 
> 
> > 
> > In this particular case, I use squid to do reverse proxy with SSL offload to
> > a MS Exchange server.
> 
> You also have a TLS connection to the exchange:443 peer.

I have both for testing, but only one is active at any one time (as 
referenced in a cache_peer_access).  Yes I now use the 443 peer because 
Chrome/IE/Edge + SSL Offloading no longer works.

> 
> Notice that in the logs you showed transactions sent to that peer get 
> the secure flag set, despite the SSL offload being done by Squid at the 
> same time.
> 
> 

Not true AFAICT.  Any cookies you see in those logs with Secure set are shown 
under the "WORKING" heading whose entries are:
2020/10/28 12:01:23.549 kid1| 11,2| http.cc(2263) sendRequest: HTTP Server local=squid-internal:62597 remote=exchange:443 FD 30 flags=1

No SSL offloading there.  Otherwise it would say "remote=exchange:80"

> 
> > Because the requests are HTTP IIS does not set the secure flag on cookies,
> > and browsers now reject them.  This breaks things.
> > 
> > I've fixed it by switching back to HTTPS on the backend (no SSL offload), and
> > so the secure flag is now being set by IIS.  Problem solved.
> > 
> > In the long term it seems doing SSL offload to MS Exchange (and Sharepoint I
> > think) will not be an option.  I've seen other proxy manufacturers provide
> > cookie manipulation; I assume for this kind of issue.
> > 
> > Do you think Squid should have such a feature?
> 
> We have ICAP, eCAP, and *_header_replace features already. So any 
> proposed new feature would have to be better than what they already provide.
> 

Unless I'm reading the online command reference incorrectly, *_header_replace 
does not allow for the insertion of text so can't help here.  Please tell me 
I'm wrong - I'd love for Squid to have this capability like other proxies.

> 
> Amos
> 

I can't tell if we're agreeing or not :).  Suffice to say SSL offloading of 
OWA with modern browsers is becoming a thing of the past, at least with Squid 
(without ICAP).

This MS article basically sums up what we've discussed here.
https://docs.microsoft.com/en-us/answers/questions/73384/chrome-80-samesite-feature-causes-owa-login-loop.html

The advice therein talks about:
- Toggling the setting in Chrome
- Do a cookie rewrite on the SSL offloader device if possible
- change the backend communication to HTTPS (turn off SSL offloading)

Anyway this conversation may help someone else.
Thanks