[squid-users] peek and splice content inspection question

[Amos Jeffries]

On 14/08/2015 9:15 a.m., Yuri Voinov wrote:
> 
> 
> 
> 14.08.15 2:56, Alex Rousskov пишет:
>> On 08/13/2015 09:38 AM, Amos Jeffries wrote:
>>> On 14/08/2015 12:47 a.m., Marko Cupać wrote:
>>>> Is it possible - by means of squid's peek and splice feature - to
>>>> inspect file extensions and mime types of https traffic? Can bumped
>>>> https traffic be forwarded to icap (squidclamav) for AV scanning?
> 
>>> Doing so is the features intended purpose.
> 
> 
>> And you may be able to use either Secure ICAP (Squid 4) or the eCAP
>> ClamAV adapter for AV scanning without transmitting bumped messages over
>> plain text ICAP connections.
> Yet another solution is not transmit any over net. Just setup all
> services on blade system or one box.
> 

Like Alex said the design of Clam' AV and toolchain is that it uses disk
storage for relaying objects between processes. There are some popular
security policies where disk storage is forbidden.


> 
>>> if I just send traffic to squidclamav on icap
>>> tcp port, then I don't store usernames and passwords or private emails
>>> in cache?
> 
>> Squid caching is not related to AV scanning. If you do not disable
>> caching, Squid will cache cachable responses. IIRC, the code making the
>> cachability decision does not check whether the response was bumped.
>> However, you may configure it to do so using the "cache" directive:
> 
>>   http://www.squid-cache.org/Doc/config/cache/

Or alternatively use a memory-only proxy cache. This allows a large
portion of the caching HIT benefits to still be gained without violating
any security requirements about persistent storage of TLS or HTTPS
message data.
 That only covers the Squid cache storage part of the system though.


> 
>> Said that, most responses with private information should not be
>> cachable by default because the server should mark them as such.
>
> ... and we ignore them due to abuse of server owners no-cache directives
> when we fight for increase hit-ratio. There is millions cache-unfriendly
> web servers, starting from Google...


No Yuri. The confusing "no-cache" control fequently used means only that
the proxy needs to revalidate the cache HIT content and headers before
delivering to the client.

All current Squid releases do that correctly. The squid.conf settings
once available to ignore/override are no longer existing.


Alex was talking of "private" and "no-store" directives. Their meaning
is clear and precise - not easily confused. Overriding those is somewhat
stupid.


> 
>> The current eCAP ClamAV adapter [temporary] stores message bodies on
>> disk to pass them to the ClamAV library for analysis. I do not know
>> about squidclamav.
> 

It seemed to do the same when I checked it a few months ago. AFAICS it
is the backend AV library only scanning disk objects that causes the
whole issue. Otherwise the eCAP could be much, much faster.

Amos