[squid-users] peek and splice and splice()

[James Harper]

> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 17/10/2014 9:47 p.m., James Harper wrote:
> > Just reading up on this, the Feature page
> > http://wiki.squid-cache.org/Features/SslPeekAndSplice says:
> >
> > "... with Squid shoveling TCP bytes back and forth without any
> > decryption"
> >
> > I can't see that squid actually uses the splice() system call, so
> > that would mean squid would actually read the data into userspace
> > then write it out again, is that right? I assume once this happens
> > it's the same code as a CONNECT.
> >
> > What if there was a way to actually splice the two TCP connections
> > at the kernel level so that Squid didn't actually need to be
> > involved anymore, except when the connection closes to do the
> > accounting? There appear to be a few historic attempts to implement
> > this in the Linux kernel but I suspect they haven't been
> > maintained.
> >
> > And would this be a worthwhile optimisation anyway or is shunting
> > data not a significant part of squid load?
> 
> There is sendfile() IIUIC. Probably worthwhile, but to do it we have
> to first drop the delay_pools feature which relies on each byte being
> accounted as it is delivered.
> 

splice() would be a reasonable compromise then - it involves squid servicing the connection each time data is available, but avoids the read-into-userspace-buffer/write-from-userspace-buffer.

> For now I am working on it from a publicity standpoint, encouraging
> people to go to the more efficient QoS controls outside of Squid. So
> the popularity of delay pools drops down to a level we can start to
> seriously consider alternatives to the current tunnel code.
> 

Yes Linux and BSD both have very good QoS, but you can only see half the connection (client<->squid or squid<->server) at that level so you can't do it for all cases.

It would be awesome if squid could intercept connections without breaking them. You can kind of do it with the ULOG/NFQUEUE target in netfilter, but you get fed packets, not streams, so you'd have to implement your own TCP stack (so... no). Such a thing would allow better outcomes in the "can't connect to server" case though.

An ideal interface would allow you to be notified of an incoming connection, reject it (icmp port unreachable, reset, etc) if you can determine that the destination is down, accept it if you can determine that the destination is up, do what you need to do, then just give it back to the kernel if no more processing is required, then be notified when the connection has finished for accounting.

James