[squid-users] Proxy Server closes the connection to http server before transferring all application data to http client

Thu Dec 17 08:31:34 UTC 2020

Hi Alex,

//If you want to work on that, respond to this message and somebody will send you more detailed instructions.//
Yes, please tell me how to do this.  

I am curious of 
1. When http client uses close, and server uses keep-alive, when connection between proxy server and http server is closed, it still can transfer data between client and proxy server, and data transfer to http client is not completed sometimes.

2. When http client uses close, and server uses closes, no data transfer from http client and proxy server after http server closes the connection.  Case 2 meets the description https://curl.se/mail/lib-2013-01/0246.html, client get error code CURLE_UNSUPPORTED_PROTOCOL which tells that connection was shutdown.

Thanks
Lily

-----Original Message-----
From: Alex Rousskov <rousskov at measurement-factory.com> 
Sent: Wednesday, December 16, 2020 10:50 PM
To: Zhang, Lily (USD); squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Proxy Server closes the connection to http server before transferring all application data to http client

[EXTERNAL EMAIL] 

On 12/16/20 4:02 AM, Zhang, Lily (USD) wrote:

> Issue still can be reproduced when http client using connection: close, and http server uses connection: keep-alive.
> I can't reproduce this issue when both side using keep-alive.

When everything works as it should, the HTTP Connection header has no effect on the data transfer during the transaction that header belongs to. Both "close" and "keep-alive" instructions tell the agents what to do _after_ the entire response body is delivered from the server to the client. This is just how HTTP works... Thus, what you are describing suggests a bug in client software, server software, Squid, and/or the test.

Going forward:

* If you do not need further assistance and do not want to work to discover the true cause of the problem, then just ignore this message. I am glad you found a workaround, and we can leave it at that.

* Otherwise, the best next step may be to share Squid debugging logs while reproducing the problem in the lab. If you want to work on that, respond to this message and somebody will send you more detailed instructions.

Cheers,

Alex.

> -----Original Message-----
> From: Zhang, Lily (USD)
> Sent: Wednesday, December 16, 2020 3:29 PM
> To: Alex Rousskov; squid-users at lists.squid-cache.org
> Subject: RE: [squid-users] Proxy Server closes the connection to http 
> server before transferring all application data to http client
> 
> Hi Alex,
> Thanks for your reply.
> 
> At first, http server turns off  keep-alive, then http client can't get the full response sometimes (example 20 0000 char, get 17 0000 chars).
> 
> I did a test in the lab, found
> 
> 1. If http client uses connection: close, and http server uses 
> connection: keep-alive, then still can transfer data after tcp 
> connection between http server and proxy server is closed.  (don't 
> know how this happens)
> 
> 2. If http client uses connection: keep-alive, and http server uses connection: keep-alive, then connection between http server and proxy server won't closed first. It is closed after client closes the connection.
> 
> Thanks
> Lily
> -----Original Message-----
> From: Alex Rousskov <rousskov at measurement-factory.com>
> Sent: Tuesday, December 15, 2020 11:53 PM
> To: squid-users at lists.squid-cache.org
> Cc: Zhang, Lily (USD)
> Subject: Re: [squid-users] Proxy Server closes the connection to http 
> server before transferring all application data to http client
> 
> 
> [EXTERNAL EMAIL]
> 
> On 12/15/20 2:52 AM, Zhang, Lily (USD) wrote:
> 
>> Looks we can use "Connection: keep-alive" in http request to solve this issue.
> 
> That client request header will not change Squid's reaction to Squid-server connection closure by the server. It also should not change the request that Squid sends to the server (if it does, it is a Squid bug). In HTTP proxying, client-Squid and Squid-server connections/negotiations are mostly independent.
> 
> If changing the client to send Connection:keep-alive to Squid fixes your problem, then the problem was most likely unrelated to the behavior you asked about earlier.
> 
> If you can configure the HTTP server to keep the connection open/persistent after delivering the response (and, hence, to send a Connection:keep-alive _response_ header or otherwise signal HTTP connection persistency), then, yes, Squid may pool that _open_ Squid-server connection for future reuse (again, regardless of what headers the HTTP client is using).
> 
> Alex.
> 
> 
> 
>> -----Original Message-----
>> From: squid-users <squid-users-bounces at lists.squid-cache.org> On 
>> Behalf Of Amos Jeffries
>> Sent: Tuesday, December 15, 2020 3:47 PM
>> To: squid-users at lists.squid-cache.org
>> Subject: Re: [squid-users] Proxy Server closes the connection to http 
>> server before transferring all application data to http client
>>
>>
>> [EXTERNAL EMAIL]
>>
>> On 15/12/20 4:21 pm, Zhang, Lily (USD) wrote:
>>> Hi
>>>
>>> I installed 4.13 squid proxy server. See attachment, http server
>>> (10.250.16.46) sends FIN, ACK to tells that response is finished. 
>>> Proxy server (10.244.102.133) sends FIN, ACK back to http server
>>> (10.250.16.46) before "Application Data" is transferred to http 
>>> client (10.245.166.20).
>>>
>>> Would you please help me on questions below:
>>>
>>>  1. Is it normal that proxy server sends FIN, ACK to http server before
>>>     http client finishes receiving  all the “Application Data” ?
>>
>> That depends on the transaction which is being performed.
>>
>> If it is a normal HTTP request-response transaction then yes. Once Squid has the response the server connection is done with - it may be closed or re-used for other transactions.
>>
>> If it is a tunnel containing non-HTTP traffic then no. Squid should only close the server connection when the client closes its end of the tunnel.
>>
>> Also, note that FIN+ACK in a single packet is a *response* to a FIN having come from the server itself. Not initiated by the proxy.
>>
>>
>>
>>>  2. Does proxy server have option to stop item 1?
>>>
>>
>> In general: No. This is something that is supposed to happen (or not) 
>> according to the relevant protocol requirements.
>>
>> In specific, there may be some options that can be configured to 
>> change how the protocol behaves. Preventing it needing a close. You 
>> will have to find out what the cause actually is to determine where 
>> to look for solutions.
>>
>>
>> Amos
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
>>
> 