[squid-dev] [PATCH] Reject responses with conflicting Content-Length

[Amos Jeffries]

On 12/08/2015 6:34 a.m., Alex Rousskov wrote:
> On 08/10/2015 11:30 PM, Amos Jeffries wrote:
> 
>> There is exactly 2 cases of benign malformation:
> ...
>> All other malformations are *malign*.
> 
> 
> This is your opinion, not a fact.
> 

Which one of these malformations is not malign ?

 * non-numeric Content-Length

 * negative value Content-Length

 * Content-Length with also Transfer-Encoding header

 * multiple different-value Content-Length

 * Content-Length on 204 response
 (<http://lists.w3.org/Archives/Public/ietf-http-wg/2015AprJun/0488.html>)

Those plus the 2 benign ones are the entire set of possible malformations.


Note that I am speaking of malformations. There are several cases were
even the above nastiness is *not* malformed;


* Wrong size value is not malformed so long as it is a positive (0 or
more) integer value, and only one Content-Length header exists.

* Content-Length on HEAD and 304 are not malformed so long as the value
is positive (0 or more) integer.
(<http://lists.w3.org/Archives/Public/ietf-http-wg/2015AprJun/0489.html>)

* CONNECT (request and reply) are not malformed, regardless of what crap
might be in the header value.
 (<http://www.rfc-editor.org/errata_search.php?rfc=7231&eid=4351>)
"
On Wed, Apr 29, 2015 at 04:00:36PM -0700, Roy T. Fielding wrote:
>> Ignoring the Content-Length has the side-effect that the 2xx CONNECT
>> response body is treated as application data.  Why ignore that which
>> must not be sent, and why ignore it if processing it and then treating
>> the response body as application data... has the same effect?
>
> The header fields MUST be ignored -- not the bits after the response.
> The response has no body!
"



> IMO, being "benign" cannot be defined by an RFC because that
> classification depends on real-world circumstances, not just syntax
> rules. An RFC can require action X when a header is malformed, of
> course. We ought to follow that recommendation except if doing X breaks
> too many benign real-world cases. If the latter happens, we may want to
> give admin a choice.
> 
> All of that complexity and decision making is outside this thread scope.
> 

I'm getting a little tired of repeating this phrase. So I'll quote :
"
On 29/04/2015 5:01 p.m., Willy Tarreau wrote
>
> Don't forget that RFC723x aim at describing
> what *is* deployed and how to best interoperate.
>
> Regards,
> Willy
>
"

Each time we go against spec we retreat from the situation of current
best interoperability.

It took more than 7 years, but these things *were* measured, *were*
tested, bugs filed and software slowly fixed (including Squid
wrong-tolerance hacks removed), then re-measured and tested.
 In the end the RFC is a concise outline of what will actually work with
todays (well, 2012-ish) software, and what won't.

A lot of the issues you have been pointing out here and elsewhere is a
result of people experimenting with their newly approved abilities.
Sometimes getting it wrong, sometimes hitting remaining Squid flaws. But
that is cause for reporting failures to the experimenters so they can
fix inline with the spec (and/or us if we are the violator like this C-L
case), not for bending over backwards to silently allow a whole new
generation of brokenness to creep in.


> 
>>> For the record, here is a Bing server response that prompted this change:
>>>
>>>> HTTP/1.1 206 Partial Content
>>>> Cache-Control: public, max-age=15552000
>>>> Content-Length: 14543
>>>> Content-Type: application/x-javascript; charset=utf-8
>>>> Last-Modified: Tue, 14 Jul 2015 01:35:04 GMT
>>>> Vary: Accept-Encoding
>>>> Server: Microsoft-IIS/8.5
>>>> Date: Wed, 05 Aug 2015 15:42:26 GMT
>>>> Content-Length: 300
>>>> Content-Range: bytes 14243-14542/14543
>>>> Accept-Ranges: bytes
>>>
>>> As you can see, the addition of an extra Content-Length field was
>>> probably triggered by the partial response to the Range request.
> 
> 
>> Is that an IIS? Bing? or ICAP server bug?
> 
> AFAIK, the quoted header did not come from the ICAP server under Squid
> control. I believe it came from some Bing server, but I do not know the
> details. The reasons I mentioned that header here is to document that
> the problem is happening right now, possibly on a relatively popular
> site, and may involve a relatively complex combination of headers.
> 

And thank you for that. I am passing it on to Mike B. to see if he can
get it fixed at their end too.


Which reminds me, if you can get similar details about the AWS software
breakage with '|' URI characters and please make sure that gets reported
somewhere appropriate. But I digress.

Amos