[squid-dev] unsupported protocol classification

[Alex Rousskov]

[ I am changing the Subject line for this sub-thread because this new
discussion is not really relevant to the unsupported protocol bypass
feature, even though that bypass feature will be used by those who need
to classify unsupported protocols. ]


On 12/31/2014 03:33 AM, Marcus Kool wrote:

> The current functionality of filtering is divided between Squid itself and
> 3rd party software (ICAP daemons and URL redirectors).

... as well as external ACLs and eCAP adapters.


> I plea for an interface where an external helper can decide what to do
> with an unknown protocol inside a tunnel because it is much more flexible
> than using ACLs and extending Squid with detection of (many) protocols.

I doubt pleading will be enough, unfortunately, because a considerable
amount of coding and design expertise is required to fulfill your dream.
IMO, a quality implementation would involve:

1. Encoding the tunnel information (including traffic) in [small]
HTTP-like messages to be passed to ICAP/eCAP services. It is important
to get this API design right while anticipating complications like
servers that speak first, agents that do not send Hellos until they hear
the other agent Hello, and fragmented Hellos. Most likely, the design
will involve two tightly linked but concurrent streams of adaptation
messages: user->Squid->origin and origin->Squid->user. Let's call that
TUNMOD, as opposed to the existing REQMOD and RESPMOD.

2. Writing adaptation hooks to pass tunnel information (using TUNMOD
design above) to adaptation services. The primary difficulty here is
handling incremental "give me more" and "give them more" decisions while
shoveling tunneled bytes. The current tunneling code does not do any
adaptation at all so the developers would be starting from scratch
(albeit with good examples available from non-tunneling code dealing
with HTTP/FTP requests and HTTP/FTP responses).

3. Implementing more actions than the already implemented "start a blind
tunnel" and "respond with an error". The "shovel this to the other side
and then come back to me with the newly received bytes" action would be
essential in many production cases, for example.

The above is a large project. I do not recall any projects of that size
and complexity implemented without sponsors in recent years but YMMV.


Please note that modern Squid already has an API that lets 3rd party
software to pick one of the supported actions. It is called annotations:
External software sends Squid an annotation and the admin configures
Squid to do X when annotation Y is received in context Z.


> A while back when we discussed the older sslBump not being able to cope
> with Skype I suggested to use ICAP so that the ICAP daemon receives a
> REQMOD/RESPMOD message with CONNECT and intercepted content, which also is
> a valid option for me.

Yes, ICAP/eCAP is the right direction here IMO, but there are several
challenges on that road. I tried to detail them above.


HTH,

Alex.