[squid-users] Squid not responding during file upload.

[Saravanan Coimbatore]

-----Original Message-----
From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Amos Jeffries
Sent: Thursday, March 19, 2015 2:26 AM
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Squid not responding during file upload.

On 19/03/2015 8:11 p.m., Saravanan Coimbatore wrote:
> Hello all, 
> 
> I am using Squid 3.4 to inspect content that heads out to the cloud from enterprise. I have two c-icap filter that does the content inspection. 
> 
> Observation: 
> - Upload 3000 1M files to cloud passes through successfully. 
> - Upload 300 40M files to cloud results in multiple failures. Some of errors: 400 Bad Request, Request Timed out.. 
> 
> Tcpdump of the 40MB file upload tests indicate the following:
> - Boto client used to upload sends packet to squid proxy. 

 Squid on receiving requests sends them to the ICAP REQMOD service,
   and waits for its response,
 then sends the ICAP REQMOD result to the origin server,
   and waits for its response,
 then sends that to the ICAP RESPMOD service,
   and waits for its response,
 then sends that to the client.

So...
 What is the ICAP service and the origin server doing?

<SC> ICAP Service inspects the data that passes through it, and does selective filtering based on user policies. There are two icap services that handles two different service providers. Would having two icap services cause any delay. Does Squid send data to a icap service if the icap service has returned 204 in the preview handler?  

The Origin server is S3 or Box. We did tests without Squid in between, and the success ratio is high. We are trying to isolate this on a component basis, but the tcp dump shows that the squid did not respond at the TCP level. We were wondering if this is because Squid is busy. 


> - Proxy does not acknowledge. 

What type of "acknowledge" are you expecting here? HTTP or TCP level?
<SC> TCP level. 

> - Client sends the data again at least 6 times, Squid does not respond. 

At TCP or HTTP layer?
<SC> At TCP Layer. 

> - After 20-25 seconds of this (where Squid did not send any data to cloud), Cloud storage vendor returns a "BAD Request" response. 
> 
> Uploading 300 files seems to be a load that should be manageable by Squid. Can anyone guide me on how to optimize Squid for the above scenario? Are there any performance parameters that I can tweak so Squid handles this correctly?
> 


Amos

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users