[squid-users] Squid not responding during file upload.

[Saravanan Coimbatore]

-----Original Message-----
From: Amos Jeffries [mailto:squid3 at treenet.co.nz] 
Sent: Thursday, March 19, 2015 11:13 PM
To: Saravanan Coimbatore; squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Squid not responding during file upload.

On 20/03/2015 6:26 a.m., Saravanan Coimbatore wrote:
> 
> From: Amos Jeffries
> 
> On 19/03/2015 8:11 p.m., Saravanan Coimbatore wrote:
>> Hello all,
>>
>> I am using Squid 3.4 to inspect content that heads out to the cloud from enterprise. I have two c-icap filter that does the content inspection. 
>>
>> Observation: 
>> - Upload 3000 1M files to cloud passes through successfully. 
>> - Upload 300 40M files to cloud results in multiple failures. Some of errors: 400 Bad Request, Request Timed out.. 
>>
>> Tcpdump of the 40MB file upload tests indicate the following:
>> - Boto client used to upload sends packet to squid proxy. 
> 
>  Squid on receiving requests sends them to the ICAP REQMOD service,
>    and waits for its response,
>  then sends the ICAP REQMOD result to the origin server,
>    and waits for its response,
>  then sends that to the ICAP RESPMOD service,
>    and waits for its response,
>  then sends that to the client.
> 
> So...
>  What is the ICAP service and the origin server doing?
> 
> <SC> ICAP Service inspects the data that passes through it, and does selective filtering based on user policies. There are two icap services that handles two different service providers. Would having two icap services cause any delay. Does Squid send data to a icap service if the icap service has returned 204 in the preview handler?  
> 

Good to know. Though I meant more along the lines of what data its been sent and what Squid got back from it etc, in each step of the above theoretical sequence of operations. If they all worked properly there would be no hang problem.


> The Origin server is S3 or Box. We did tests without Squid in between, and the success ratio is high. We are trying to isolate this on a component basis, but the tcp dump shows that the squid did not respond at the TCP level. We were wondering if this is because Squid is busy. 
> 
> 
>> - Proxy does not acknowledge. 
> 
> What type of "acknowledge" are you expecting here? HTTP or TCP level?
> <SC> TCP level. 
> 

Aha. That only happens if the receive buffer in Squid is full, or Squid believes the request is fully received and is now waiting for the response data for it to deliver.

<SC> Is there a debug message that will be logged when Squid receive buffer is full?
<SC> What config parameter that we can use to increase Squid receive buffer?
Amos