[squid-users] Need help about ICAP scan timeout/max file size for big files

[Alex Rousskov]

On 1/8/19 9:46 AM, info at schroeffu.ch wrote:

> With ClamAV C-ICAP there is defined "MaxStreamSize 25M" as default,
> so after 25MB scanned by ICAP I can see with tcpdump on port 1344
> "ICAP/1.0 200 OK" from ICAP to Squid which triggers the browser to
> start the download. Thats what i want also for F-Secure ICAP.

The best solution would be for F-Secure to add support for (or enable in
your setup) "data trickling" or "patience pages". Any workarounds inside
Squid would be either nasty (e.g., timeouts, abandoned transactions,
etc.) or expensive (require Squid or eCAP/ICAP wrapper development).


> if their ICAP really is not sending "ICAP/1.0 200 OK" after X
> Seconds/MB, can I configure SQUID with a workaround?

You can try to specify a timeout via icap_io_timeout. Bugs
notwithstanding, Squid would terminate a connection to the ICAP service
that does not respond in X seconds. You may need to adjust
icap_service_failure_limit and/or icap_service_revival_delay to avoid
marking the affected ICAP service as "down" [too often]. Again, this is
not a proper solution and it may have negative side effects such as
memory leaks and unresponsive ICAP service. It may be worth trying while
you wait for F-Secure.

Unfortunately, the icap_io_timeout may not work if Squid is constantly
writing to the ICAP service (to deliver more virgin body bytes). Squid
should be treating each such write as an I/O, resetting the timeout.


You can also hack Squid to treat these cases specially. For example, you
could add adaptation_response_timeout or a similar directive that would
work like icap_io_timeout but ignore write activity. If you go down that
route, I suggest posting an RFC with new option description to squid-dev
as the first step.


You can even write an ICAP service (or eCAP adapter) that will add data
trickling or patience pages support to any ICAP service, but that is a
lot of development work!


> The header seems not include the file size. Here is an example of
> 100MB Virus File

Please note that you should test/analyze "real" transactions, not
requests for test files. If real transactions of interest usually lack
the Content-Length header, then timeout-based knobs are your best bet
(see above): There are no ACLs that can match accumulated response size
and, more importantly, there is no directive that would repeatedly
evaluate such ACLs as Squid accumulates the response body while waiting
for the ICAP response.


HTH,

Alex.