[squid-dev] unsupported protocol classification

[Marcus Kool]

On 12/31/2014 02:23 PM, Alex Rousskov wrote:
> [ I am changing the Subject line for this sub-thread because this new
> discussion is not really relevant to the unsupported protocol bypass
> feature, even though that bypass feature will be used by those who need
> to classify unsupported protocols. ]
>
>
> On 12/31/2014 03:33 AM, Marcus Kool wrote:
>
>> The current functionality of filtering is divided between Squid itself and
>> 3rd party software (ICAP daemons and URL redirectors).
>
> ... as well as external ACLs and eCAP adapters.
>
>
>> I plea for an interface where an external helper can decide what to do
>> with an unknown protocol inside a tunnel because it is much more flexible
>> than using ACLs and extending Squid with detection of (many) protocols.
>
> I doubt pleading will be enough, unfortunately, because a considerable
> amount of coding and design expertise is required to fulfill your dream.
> IMO, a quality implementation would involve:

It is clear to me that this functionality will not be implemented next week,
but for me it is not a dream.  It is a reality that filtering becomes more
important, just wait until a headline in the news comes along like
"secret document stolen via a web tunnel" and everybody wants it.
The risk is real and it is so simple to abuse CONNECT on port 443 for anything
that it is extremely likely that it is already being used for illegal actions
and will continue to be used for illegal actions.

There is also not much point in having a web proxy that can filter 50% or 99%
of what you want to filter.  If you cannot filter everything and especially cannot
filter known security risks, the filter solution is very weak.
That is why ufdbGuard currently sends probes to sites that an application CONNECTs to.
The probes tell ufdbGuard what type of traffic is to be expected but
are also not 100% reliable since a probe is not the same as an inspection
of the real traffic.

> 1. Encoding the tunnel information (including traffic) in [small]
> HTTP-like messages to be passed to ICAP/eCAP services. It is important
> to get this API design right while anticipating complications like
> servers that speak first, agents that do not send Hellos until they hear
> the other agent Hello, and fragmented Hellos. Most likely, the design
> will involve two tightly linked but concurrent streams of adaptation
> messages: user->Squid->origin and origin->Squid->user. Let's call that
> TUNMOD, as opposed to the existing REQMOD and RESPMOD.

Getting the design right is definitely important.  Therefore I like
to bring up this issue once in a while so that with the design decisions
made today of related parts, it will be easier to implement TUNMOD
in the future.

> 2. Writing adaptation hooks to pass tunnel information (using TUNMOD
> design above) to adaptation services. The primary difficulty here is
> handling incremental "give me more" and "give them more" decisions while
> shoveling tunneled bytes. The current tunneling code does not do any
> adaptation at all so the developers would be starting from scratch
> (albeit with good examples available from non-tunneling code dealing
> with HTTP/FTP requests and HTTP/FTP responses).

It can be simpler.  TUNMOD replies can be limited to
DONTKNOW - continue with what is happening and keep the TUNMOD server informed
ACCEPT - continue and do not inform the TUNMOD server any more about this tunnel
BLOCK - close the tunnel

I think there is no need for adaptation since one accepts a webdisk, voice
chat, VPN or whatever, or one does not accept it. So adaptation as is
used for HTTP, is not an important feature.

Sending an HTTP error on a tunnel is only useful if the tunnel uses
SSL-encapsulated HTTP.

> 3. Implementing more actions than the already implemented "start a blind
> tunnel" and "respond with an error". The "shovel this to the other side
> and then come back to me with the newly received bytes" action would be
> essential in many production cases, for example.
>
> The above is a large project. I do not recall any projects of that size
> and complexity implemented without sponsors in recent years but YMMV.

We will see.  Maybe there will be a sponsor to do this.

It is 15:38 local time and my last post of the year.
Happy New Year to all.

Marcus

> Please note that modern Squid already has an API that lets 3rd party
> software to pick one of the supported actions. It is called annotations:
> External software sends Squid an annotation and the admin configures
> Squid to do X when annotation Y is received in context Z.
>
>
>> A while back when we discussed the older sslBump not being able to cope
>> with Skype I suggested to use ICAP so that the ICAP daemon receives a
>> REQMOD/RESPMOD message with CONNECT and intercepted content, which also is
>> a valid option for me.
>
> Yes, ICAP/eCAP is the right direction here IMO, but there are several
> challenges on that road. I tried to detail them above.
>
>
> HTH,
>
> Alex.
>
>