[squid-users] Bad HTTP requests trigger ICAP suspension

[Amos Jeffries]

On 5/12/2016 11:17 p.m., Silamael wrote:
> Hi,
> 
> We are using the ICAP services of Squid for filtering HTTP-Requests. Now
> we encountered the problem that a buggy? web application creates
> requests with an URL with the hostname set to '.'.

Ok.

> These bad requests than cause the suspension of the whole ICAP service
> which then causes the bypass of the URL filtering as the ICAP service
> has bypass=yes configured.

These are not exactly bad requests. '.' is a valid URL host segment.
Very much not likely to succeed in todays Internet, but still valid.

In DNS terms it means the root zone server is the origin host. However,
there is no requirement that hostnames resolve using DNS. So it may have
another meaning if passed to a service that resolves it differently (eg.
the /etc/hosts mapping, or your ICAP service).

There may or may not be bugs in the ICAP service itself handling that
weird input. I can't speak for that software.


> This sounds somehow wrong to me, the ICAP service doesn't have a
> problem, just the HTTP request being forwarded is borken. Therefor is no

The ICAP service appears to be producing URLs without any host segment
at all ("") instead of using the input "." or some other value of its
choosing.

Strictly speaking Squid should accept that as a URL with no hostname,
just like it accepted the odd '.' hostname. That is one of the issues
bug 1961 seeks to fix.


> need to suspend the ICAP service at a whole for 30 seconds.
> Am I wrong or is this behavior intended?
> 

AIUI, for installations using bypass=yes the behaviour is expected. That
setting is not tied to particular types of error, it simply means the
service is optional. *any* HTTP request (even good ones) can bypass.


There is an interoperability issue between Squid and the ICAP service
causing the transaction to break down. The proper handling seems to
happen after that.


> This are the log messages:
> 2016/11/28 13:30:00 kid1| SECURITY ALERT: Missing hostname in URL
> 'http:///?_task=mail&_id=xxxxxxxxxxx&_action=display-attachment&_file=rcmfilexxxxxxxxxxxxxxxxxxxxx'.
> see access.log for details.
> 2016/11/28 13:30:00.740 kid1| 0,3| src/base/TextException.cc(87) Throw:
> src/adaptation/icap/ModXact.cc:970:exception:
> adapted.header->parse(&httpBuf, true, &error)
> 2016/11/28 13:30:00.740 kid1| suspending ICAP service for too many failures
> 2016/11/28 13:30:00.740 kid1| optional ICAP service is suspended:
> icap://XXX.XXX.XXX.XXX:1344/reqmod [down,susp,fail11]
> 

IIRC that "fail11" means the service repeatedly failed (11 times) before
it got suspended for 30sec. Presumably those were 10 bypasses and one
final 'I give up -> suspend' from Squid.

Amos