[squid-dev] [squid-users] tcp_outgoing_address and HTTPS

[Amos Jeffries]

On 21/03/18 00:11, Michael Pro wrote:
> squid-5 master branch, not have  personal/private repository changes,
> not use  cache_peer's ability, (if it's matters - not use transparent
> proxying ability).
> 
> We have a set of rules (ACL's with url regex) for content, depending
> on which we make a decision for the outgoing address, for example,
> from 10.10.1.xx to 10.10.6.xx
> -----log 1part {{{ -----
> Acl.cc(151) matches: checked: tcp_outgoing_address 10.10.5.11 = 1
> Checklist.cc(63) markFinished: 0x7fffffffe2b8 answer ALLOWED for match
> FilledChecklist.cc(67) ~ACLFilledChecklist: ACLFilledChecklist
> destroyed 0x7fffffffe2b8
> Checklist.cc(197) ~ACLChecklist: ACLChecklist::~ACLChecklist:
> destroyed 0x7fffffffe2b8
> peer_select.cc(1026) handlePath: PeerSelector3438 found
> local=10.10.5.11 remote=17.253.37.204:80 HIER_DIRECT flags=1,
> destination #2 for http://iosapps.itunes.apple.com/...xxx...ipa
> ...
> peer_select.cc(1002) interestedInitiator: PeerSelector3438
> peer_select.cc(112) ~PeerSelector: http://iosapps.itunes.apple.com/...xxx...ipa
> store.cc(464) unlock: peerSelect unlocking key
> 60080000000000001C0E000001000000 e:=p2IWV/0x815c09500*3
> AsyncCallQueue.cc(55) fireNext: entering AsyncJob::start()
> AsyncCall.cc(38) make: make call AsyncJob::start [call195753]
> AsyncJob.cc(123) callStart: Comm::ConnOpener status in: [ job10909]
> comm.cc(348) comm_openex: comm_openex: Attempt open socket for: 10.10.5.11
> comm.cc(391) comm_openex: comm_openex: Opened socket local=10.10.5.11
> remote=[::] FD 114 flags=1 : family=2, type=1, protocol=6
> -----log 1part }}} -----
> In the case of normal traffic (http), everything works fine, as shuld.
> 

The difference to be aware of is that there is zero security on this
type of HTTP. So while it is better not to play with destinations, and
Squid default is to go where the client wanted - it is permitted to go
elsewhere if a better source is found.


> In the case of HTTPS with traffic analysis (ssl_bump) we have such a picture:
> -----log 2part {{{ ------
> Acl.cc(151) matches: checked: tcp_outgoing_address 10.10.5.120 = 1
> Checklist.cc(63) markFinished: 0x7fffffffe2b8 answer ALLOWED for match
> FilledChecklist.cc(67) ~ACLFilledChecklist: ACLFilledChecklist
> destroyed 0x7fffffffe2b8
> Checklist.cc(197) ~ACLChecklist: ACLChecklist::~ACLChecklist:
> destroyed 0x7fffffffe2b8
> peer_select.cc(1026) handlePath: PeerSelector569 found
> local=10.10.5.120 remote=23.16.9.11:443 PINNED flags=1, destination #1
> for https://some.https.com/...xxx...zip

What PINNED means to Squid is that the client TCP connection is tied up
with some details related to some specific TCP server connection.

In this case the TLS crypto used during the bumping process took crypto
details from the client connection and gave them to the server, then
from the server and gave them to the client. Resulting in a forced
end-to-end relationship between the clinet and server for all traffic
over both those connections.
 The only thing Squid can do is to server some content from cache as
normal HITs, or if you specifically configure ICAP/eCAP service they can
modify the messages as they flow. Delivering the traffic to another
server is not permissable because the HTTP messages can (and often are)
tied to the TLS crypto details as well in ways that are not visible to
Squid.

For example; it is becoming very popular for the endpoints to crypto
sign messages or embed a hash signature which can only be verified valid
using details the server and client exchanged up front. No other server
would be able to send valid values and the client breaks if it is wrong.
 This kind of thing survives even when SSL-Bump'ing because of Squid
pinning, but does add the restrictions you found.


> 
> I understand that without analyzing the traffic and not knowing the
> final goal for the beginning, we can not manage the process further.
> Question: how can we break the established channel (unpinn it) along
> the old route and establish a new channel along the new route, when we
> already know how.

There are three possibilities that I am aware of - in no particular order:

1) An ICAP service can do whatever it pleases with requests it receives.
We hold no responsibility for anything happening there and I publicly
advise against playing with the crypto that way - the above issues are
the least of the problems to be faced.


2) It is technically possible to make Squid open a CONNECT tunnel
through an HTTP peer proxy to the origin instead of going there
directly. The only thing preventing this is nobody writing the necessary
code.

It has been on my (and many others) wishlist for a long while but still
nobody has been able to work on it. Any assistance towards getting that
coded is very, very welcome.


3) The client-first type of bumping does not involve any server crypto.
This is *highly* unsafe and often encounters problems like the ICAP
approach and for all the same reasons.

BUT that said, if you are sufficiently in control of the traffic to be
sure of its safety then Squid can do it by simply using the bump action
at "step 1" of the SSL-Bump process - rather than the recommended step 2
or 3.

We have err'ed on the side of security with these things so it may not
work right now. But it is not technically required to pin for this
specific type of bumped connection - so the case could be made that it
is a bug to fix if Squid does force pinning on that traffic.

Amos