[squid-users] Squid returns 400 to GET / HTTP/1.1 with Host Header

[Amos Jeffries]

On 24/04/18 03:15, Stephen Nelson-Smith wrote:
> Hello,
> 
> I need to demonstrate and test a Squid setup, which should blacklist
> by default, and allow requests only to whitelisted URLs from known
> networks. This is currently running in my staging environment, and is
> working as expected, but I want to test and demo it on demand, with
> nicer feedback than with curl.
> 
> I've deployed Redbot (https://github.com/mnot/redbot), which I've set
> up to send all HTTP requests via the Squid proxy
> 
> Using curl -x from the Redbot machine, all my tests pass, but using
> the application, Squid returns a 400 whatever happens. All requests go
> to Squid, and I see every request, but instead of returning a 403 and
> X-Squid-Error: ERR_ACCESS_DENIED 0, or allowing the request, every
> request gets a 400, and X-Squid-Error: ERR_INVALID_URL 0.
> 

ERR_INVALID_URL --> the URL is the invalid part, not the Host header.

> Digging into it - logs and tcpdump - the key difference I see is that
> Redbot sends a request of the form:
> 
> GET / HTTP/1.1
> Host: chess.com
> 
> Curl sends:
> 
> GET http://chess.com/ HTTP/1.1
> Host: chess.com
> 
> From the RFC it seems like Redbot's request is perfectly valid, and so
> I feel like Squid should do the right thing and deduce from the host
> header what Redbot wants, and go through its ACLs. However, it just
> errors with:

You missed the part where it says which type of recipient the various
URL forms are valid.

The redbot example is a origin-form URL - valid only when sent to origin
servers (or reverse-proxy). The curl one is an absolute-form URL - valid
when sent to proxies and gateways.

...
> 
> Does this seem like a Squid config issue? Or do I need to make Redbot
> make a request like Curl does?

Redbot is designed primarily for debugging HTTP problems with origin
servers to check why their output is not caching in a proxy or browser
properly. If you can find an option to inform it that it is operating
through a proxy, turn that on.

Amos