[squid-users] squid 5.7 FTP upload fails

[Alex Rousskov]

On 2024-08-13 09:53, Matus UHLAR - fantomas wrote:
>>> On 2024-08-09 09:03, Matus UHLAR - fantomas wrote:
>>>> FTR, I sent the info privately, but we can continue 
>>>> discussing/solving it here.
>>>
>>>> maybeMakeSpaceAvailable: request buffer full: 
>>>> client_request_buffer_max_size=524288
>>>> ReadNow: ... size 0, retval 0, errno 0
>>>> terminateAll: 1/1 after ERR_CLIENT_GONE/WITH_CLIENT
> 
>> On 09.08.24 09:51, Alex Rousskov wrote:
>>> AFAICT, you are suffering from a Squid bug that goes back to 2015 
>>> commit 4d1376d7 (that was fixing Squid bug 4206) or even earlier 
>>> code. Red flags were raised back in 2015, but nobody followed 
>>> through. We need to refactor 
>>> Server::maybeMakeSpaceAvailable()-related code to fix this bug.
>>>
>>> FWIW, we did similar work for Squid-to-peer communication in commit 
>>> cc8b26f9 and commit 50c5af88. Similar principles should apply to 
>>> client-Squid communication. Unfortunately, I do not have enough free 
>>> time to volunteer to fix this client-Squid bug now.
>>>
>>> At the expense of using more memory for some upload cases, you may be 
>>> able to work around the bug in some (but not all!) cases by making 
>>> client_request_buffer_max_size sufficiently large. For example, 
>>> setting client_request_buffer_max_size to 16 megabytes may allow you 
>>> to pass the test case you have privately shared.
> 
> On 12.08.24 11:27, Matus UHLAR - fantomas wrote:
>> I have submitted bug 5405 so it's documented:
>>
>> https://bugs.squid-cache.org/show_bug.cgi?id=5405
> 
> Does this bug apply for POST requests as well?

I speculate that the answer is "yes": IIRC, the underlying bug may 
affect any request with a large-enough body in _combination_ with other 
events/factors that affect buffered data consumption. I have not 
checked, but one of those factors could be, for example, the receiving 
port protocol (i.e. ftp_port and FTP in your test case) and/or REQMOD 
adaptation. I speculate that those "other" events rarely happen in a 
typical Squid installation (explaining why most large requests do not 
get stuck), but can be reproduced under controlled conditions with 
http_port POSTs.


HTH,

Alex.

> Because the biggest PUT is about 4MB so far, but I've seen POST requests 
> nearly 54MB big.