<div dir="ltr">Can you describe the process of option 2?<div><br><div>Terminate Squid-to-client transmission while indicating (to that same<br>client) that the HTTP response body was cut short (i.e. that the<br>delivery of the response was aborted). </div><div><br></div><div>What TCP Flag or ICAP header should the ICAP server send in order to inform connection aborted?? <br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Nov 27, 2019 at 12:44 PM Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com">rousskov@measurement-factory.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 11/27/19 11:01 AM, Felipe Arturo Polanco wrote:<br>
<br>
> How can we then terminate an ICAP 200 OK transaction to squid without<br>
> sending the complete body back to it? We don't want squid to mark an<br>
> ICAP failure on us if we just close the TCP connection.<br>
<br>
To properly answer your question, I have to come back to the question I<br>
asked earlier: What do you want Squid to do when your ICAP service finds<br>
a virus after trickling a few virgin HTTP body bytes to the HTTP client?<br>
<br>
The "we want Squid to send an HTTP 307 redirect to the client" desire<br>
you responded with earlier was ruled impossible to satisfy in your<br>
trickling environment. A few realistic options remain though, including:<br>
<br>
1. Terminate Squid-to-client transmission as if the whole virgin HTTP<br>
response body was sent to the client.<br>
<br>
2. Terminate Squid-to-client transmission while indicating (to that same<br>
client) that the HTTP response body was cut short (i.e. that the<br>
delivery of the response was aborted).<br>
<br>
3. #2 plus annotate the transaction specially in Squid access.log and/or<br>
detect this special outcome using Squid ACLs.<br>
<br>
Alex.<br>
<br>
<br>
> On Wed, Nov 27, 2019 at 10:54 AM Alex Rousskov wrote:<br>
> <br>
> On 11/26/19 4:12 PM, Felipe Arturo Polanco wrote:<br>
> <br>
> > The flow is the following:<br>
> > ICAP transaction is sent to ICAP server with a PREVIEW header<br>
> > ICAP server sends ICAP header 100 Continue<br>
> > ICAP server sends ICAP header 200 OK to start data transfer<br>
> <br>
> (*) Your notes below imply that the ICAP server also sends the embedded<br>
> HTTP message header back to Squid (and not just the ICAP 200 header).<br>
> <br>
> <br>
> > <data transfer begins><br>
> > ICAP server receives a chunk, checks if its the last chunk, if not<br>
> then<br>
> > append to temp file and send it back to Squid;<br>
> <br>
> The "send it back to Squid" part implies that not only your ICAP server<br>
> sent an ICAP 200 response header, but it sent the HTTP message header as<br>
> well. See (*) above. Once the ICAP server sent the HTTP message header,<br>
> it is committed to finish sending that HTTP message (and nothing else).<br>
> <br>
> <br>
> > if it is the last chunk then analyze the temp file for virus.<br>
> > <repeat for next data transfer><br>
> > If virus found then send encapsulated HTTP header 307 redirect.<br>
> <br>
> This is an ICAP service bug: One cannot send a second encapsulated HTTP<br>
> message in one ICAP response. A single ICAP response cannot contain<br>
> multiple HTTP messages. The ICAP protocol does not allow for that, and<br>
> there would be no way to actually support something like that in an ICAP<br>
> client because the HTTP message cannot be interrupted and replaced with<br>
> another mid-flight. You may assume that the HTTP client is already<br>
> seeing the beginning of the first HTTP response. The train has left the<br>
> station.<br>
> <br>
> If the above is accurate, and you are using the ICAP service correctly,<br>
> then the ICAP service that allowed you to do the above is badly broken,<br>
> probably written by somebody who thought that ICAP is "easy". You may be<br>
> better off reusing an existing higher-quality ICAP service instead.<br>
> <br>
> <br>
> > If virus not found, send the last chunk to squid.<br>
> ><br>
> > The part where we send 307 is the part that Squid doesn't like, I<br>
> > believe is because we are not sending the last chunk since the<br>
> file is a<br>
> > virus.<br>
> <br>
> Not exactly: Even if you send the last HTTP chunk, you would not be able<br>
> to follow up with an HTTP 307 message. The HTTP fate of this transaction<br>
> was sealed when you started trickling virgin HTTP message pieces back to<br>
> Squid (and, hence, back to the HTTP client talking to Squid).<br>
> <br>
> <br>
> HTH,<br>
> <br>
> Alex.<br>
> <br>
> <br>
> > On Tue, Nov 26, 2019 at 4:52 PM Alex Rousskov wrote:<br>
> ><br>
> > On 11/26/19 2:52 PM, Felipe Arturo Polanco wrote:<br>
> ><br>
> > > We are sending an encapsulated HTTP 307 redirect webpage header<br>
> > whenever<br>
> > > a Virus is found and stop sending any other data after that<br>
> ><br>
> > You must use ICAP status code 200 then. Make sure your<br>
> encapsulated HTTP<br>
> > 307 body (if any) is properly sent to Squid.<br>
> ><br>
> ><br>
> > > but squid<br>
> > > complains about ICAP failure when we do that:<br>
> > > Adaptation::Icap::Xaction::noteCommRead threw exception:<br>
> > corrupted chunk<br>
> > > size<br>
> ><br>
> > What chunk size did Squid not like? You should be able to tell by<br>
> > looking at the packet capture of the failed transaction (or<br>
> low-level<br>
> > Squid debugging).<br>
> ><br>
> ><br>
> > > We are not sending an ICAP header at this point because we<br>
> > already told<br>
> > > Squid ICAP 200 OK header and begun a body transaction, we<br>
> send some<br>
> > > chunks back to the client for progress and hold the last<br>
> part for<br>
> > scanning.<br>
> ><br>
> > Are you sending HTTP 307 body chunks to Squid? How do you<br>
> indicate that<br>
> > no more chunks will be coming?<br>
> ><br>
> > It sounds like you are trying to cram two HTTP messages (one<br>
> with the<br>
> > original HTTP response body prefix and one with a generated 307<br>
> > redirect) into one ICAP response, which is impossible, but<br>
> perhaps I<br>
> > misunderstood your description. It would help if you post a<br>
> sample (but<br>
> > complete) ICAP response that Squid does not like.<br>
> ><br>
> ><br>
> > > Ideally, we would like to just send our 307 to Squid and not<br>
> > having it<br>
> > > count as a failure.<br>
> ><br>
> > Yes, a 200 ICAP response with an embedded HTTP 307 response<br>
> should work<br>
> > just fine, but all its pieces should be properly formed (and there<br>
> > should be no extras).<br>
> ><br>
> > Alex.<br>
> ><br>
> ><br>
> > > On Tue, Nov 26, 2019 at 3:44 PM Alex Rousskov wrote:<br>
> > ><br>
> > > On 11/26/19 10:15 AM, Felipe Arturo Polanco wrote:<br>
> > ><br>
> > > > While we can successfully scan our files and do content<br>
> > adaptation, we<br>
> > > > have been struggling to find a way to close the ICAP<br>
> > transaction<br>
> > > before<br>
> > > > passing the whole body back to squid and at the same time<br>
> > avoid squid<br>
> > > > marking one icap failure.<br>
> > ><br>
> > > Squid needs a valid ICAP response. The right ICAP response<br>
> > status code<br>
> > > depends on what you want Squid to do after receiving that<br>
> > response. You<br>
> > > have mentioned what you do _not_ want Squid to do (i.e.<br>
> > increase the<br>
> > > failure count), but that still leaves a lot of options.<br>
> > ><br>
> > ><br>
> > > > This is for an ICAP server that does Virus scanning<br>
> and if<br>
> > virus<br>
> > > found,<br>
> > > > the body is not sent back.<br>
> > ><br>
> > > What do you want Squid to do when the ICAP service finds a<br>
> > virus? For<br>
> > > example, what message do you want Squid to send to the next<br>
> > HTTP hop?<br>
> > ><br>
> > > Alex.<br>
> > ><br>
> ><br>
> <br>
<br>
</blockquote></div>