[squid-users] Proxy Server closes the connection to http server before transferring all application data to http client

Thu Dec 17 17:33:02 UTC 2020

On 12/17/20 3:31 AM, Zhang, Lily (USD) wrote:

>> If you want to work on that, respond to this message and somebody will send you more detailed instructions.

> Yes, please tell me how to do this.  

Please try the procedure described at
https://wiki.squid-cache.org/SquidFaq/BugReporting#Debugging_a_single_transaction

In your case, if possible, use a single problematic master transaction
(including client-Squid and Squid-server HTTP transactions) -- the one
that results in response truncation. I recommend "ALL,9" debugging for
this case. Please share a link to the _compressed_ log.

> I am curious of 

> 1. When http client uses close, and server uses keep-alive, when
> connection between proxy server and http server is closed, it still
> can transfer data between client and proxy server,

Yes, Squid should send the rest of the response to the client after the
server sent the entire response to Squid.

Whether the server closed the connection after sending the entire
response is irrelevant (except for special cases where the end of
response is determined by the connection closure).

> and data transfer to http client is not completed sometimes.

That is a mystery you want to solve.

> 2. When http client uses close, and server uses closes, no data
> transfer from http client and proxy server after http server closes
> the connection.

I assume that by "from http client and proxy" you meant "to http client
from proxy".

That is another mystery you want to solve.

Hopefully, debugging logs will help us solve both mysteries.

Alex.

> -----Original Message-----
> From: Alex Rousskov <rousskov at measurement-factory.com> 
> Sent: Wednesday, December 16, 2020 10:50 PM
> To: Zhang, Lily (USD); squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] Proxy Server closes the connection to http server before transferring all application data to http client
> 
> 
> [EXTERNAL EMAIL] 
> 
> On 12/16/20 4:02 AM, Zhang, Lily (USD) wrote:
> 
>> Issue still can be reproduced when http client using connection: close, and http server uses connection: keep-alive.
>> I can't reproduce this issue when both side using keep-alive.
> 
> When everything works as it should, the HTTP Connection header has no effect on the data transfer during the transaction that header belongs to. Both "close" and "keep-alive" instructions tell the agents what to do _after_ the entire response body is delivered from the server to the client. This is just how HTTP works... Thus, what you are describing suggests a bug in client software, server software, Squid, and/or the test.
> 
> Going forward:
> 
> * If you do not need further assistance and do not want to work to discover the true cause of the problem, then just ignore this message. I am glad you found a workaround, and we can leave it at that.
> 
> * Otherwise, the best next step may be to share Squid debugging logs while reproducing the problem in the lab. If you want to work on that, respond to this message and somebody will send you more detailed instructions.
> 
> 
> Cheers,
> 
> Alex.
> 
> 
>> -----Original Message-----
>> From: Zhang, Lily (USD)
>> Sent: Wednesday, December 16, 2020 3:29 PM
>> To: Alex Rousskov; squid-users at lists.squid-cache.org
>> Subject: RE: [squid-users] Proxy Server closes the connection to http 
>> server before transferring all application data to http client
>>
>> Hi Alex,
>> Thanks for your reply.
>>
>> At first, http server turns off  keep-alive, then http client can't get the full response sometimes (example 20 0000 char, get 17 0000 chars).
>>
>> I did a test in the lab, found
>>
>> 1. If http client uses connection: close, and http server uses 
>> connection: keep-alive, then still can transfer data after tcp 
>> connection between http server and proxy server is closed.  (don't 
>> know how this happens)
>>
>> 2. If http client uses connection: keep-alive, and http server uses connection: keep-alive, then connection between http server and proxy server won't closed first. It is closed after client closes the connection.
>>
>> Thanks
>> Lily
>> -----Original Message-----
>> From: Alex Rousskov <rousskov at measurement-factory.com>
>> Sent: Tuesday, December 15, 2020 11:53 PM
>> To: squid-users at lists.squid-cache.org
>> Cc: Zhang, Lily (USD)
>> Subject: Re: [squid-users] Proxy Server closes the connection to http 
>> server before transferring all application data to http client
>>
>>
>> [EXTERNAL EMAIL]
>>
>> On 12/15/20 2:52 AM, Zhang, Lily (USD) wrote:
>>
>>> Looks we can use "Connection: keep-alive" in http request to solve this issue.
>>
>> That client request header will not change Squid's reaction to Squid-server connection closure by the server. It also should not change the request that Squid sends to the server (if it does, it is a Squid bug). In HTTP proxying, client-Squid and Squid-server connections/negotiations are mostly independent.
>>
>> If changing the client to send Connection:keep-alive to Squid fixes your problem, then the problem was most likely unrelated to the behavior you asked about earlier.
>>
>> If you can configure the HTTP server to keep the connection open/persistent after delivering the response (and, hence, to send a Connection:keep-alive _response_ header or otherwise signal HTTP connection persistency), then, yes, Squid may pool that _open_ Squid-server connection for future reuse (again, regardless of what headers the HTTP client is using).
>>
>> Alex.
>>
>>
>>
>>> -----Original Message-----
>>> From: squid-users <squid-users-bounces at lists.squid-cache.org> On 
>>> Behalf Of Amos Jeffries
>>> Sent: Tuesday, December 15, 2020 3:47 PM
>>> To: squid-users at lists.squid-cache.org
>>> Subject: Re: [squid-users] Proxy Server closes the connection to http 
>>> server before transferring all application data to http client
>>>
>>>
>>> [EXTERNAL EMAIL]
>>>
>>> On 15/12/20 4:21 pm, Zhang, Lily (USD) wrote:
>>>> Hi
>>>>
>>>> I installed 4.13 squid proxy server. See attachment, http server
>>>> (10.250.16.46) sends FIN, ACK to tells that response is finished. 
>>>> Proxy server (10.244.102.133) sends FIN, ACK back to http server
>>>> (10.250.16.46) before "Application Data" is transferred to http 
>>>> client (10.245.166.20).
>>>>
>>>> Would you please help me on questions below:
>>>>
>>>>  1. Is it normal that proxy server sends FIN, ACK to http server before
>>>>     http client finishes receiving  all the “Application Data” ?
>>>
>>> That depends on the transaction which is being performed.
>>>
>>> If it is a normal HTTP request-response transaction then yes. Once Squid has the response the server connection is done with - it may be closed or re-used for other transactions.
>>>
>>> If it is a tunnel containing non-HTTP traffic then no. Squid should only close the server connection when the client closes its end of the tunnel.
>>>
>>> Also, note that FIN+ACK in a single packet is a *response* to a FIN having come from the server itself. Not initiated by the proxy.
>>>
>>>
>>>
>>>>  2. Does proxy server have option to stop item 1?
>>>>
>>>
>>> In general: No. This is something that is supposed to happen (or not) 
>>> according to the relevant protocol requirements.
>>>
>>> In specific, there may be some options that can be configured to 
>>> change how the protocol behaves. Preventing it needing a close. You 
>>> will have to find out what the cause actually is to determine where 
>>> to look for solutions.
>>>
>>>
>>> Amos
>>>
>>> _______________________________________________
>>> squid-users mailing list
>>> squid-users at lists.squid-cache.org
>>> http://lists.squid-cache.org/listinfo/squid-users
>>> _______________________________________________
>>> squid-users mailing list
>>> squid-users at lists.squid-cache.org
>>> http://lists.squid-cache.org/listinfo/squid-users
>>>
>>
> 