[squid-users] What is the proper way to close an ICAP transaction?

[Felipe Arturo Polanco]

Thanks for the detailed response Alex, this is very helpful.

How can we then terminate an ICAP 200 OK transaction to squid without
sending the complete body back to it? We don't want squid to mark an ICAP
failure on us if we just close the TCP connection.

On Wed, Nov 27, 2019 at 10:54 AM Alex Rousskov <
rousskov at measurement-factory.com> wrote:

> On 11/26/19 4:12 PM, Felipe Arturo Polanco wrote:
>
> > The flow is the following:
> > ICAP transaction is sent to ICAP server with a PREVIEW header
> > ICAP server sends ICAP header 100 Continue
> > ICAP server sends ICAP header 200 OK to start data transfer
>
> (*) Your notes below imply that the ICAP server also sends the embedded
> HTTP message header back to Squid (and not just the ICAP 200 header).
>
>
> > <data transfer begins>
> > ICAP server receives a chunk, checks if its the last chunk, if not then
> > append to temp file and send it back to Squid;
>
> The "send it back to Squid" part implies that not only your ICAP server
> sent an ICAP 200 response header, but it sent the HTTP message header as
> well. See (*) above. Once the ICAP server sent the HTTP message header,
> it is committed to finish sending that HTTP message (and nothing else).
>
>
> > if it is the last chunk then analyze the temp file for virus.
> > <repeat for next data transfer>
> > If virus found then send encapsulated HTTP header 307 redirect.
>
> This is an ICAP service bug: One cannot send a second encapsulated HTTP
> message in one ICAP response. A single ICAP response cannot contain
> multiple HTTP messages. The ICAP protocol does not allow for that, and
> there would be no way to actually support something like that in an ICAP
> client because the HTTP message cannot be interrupted and replaced with
> another mid-flight. You may assume that the HTTP client is already
> seeing the beginning of the first HTTP response. The train has left the
> station.
>
> If the above is accurate, and you are using the ICAP service correctly,
> then the ICAP service that allowed you to do the above is badly broken,
> probably written by somebody who thought that ICAP is "easy". You may be
> better off reusing an existing higher-quality ICAP service instead.
>
>
> > If virus not found, send the last chunk to squid.
> >
> > The part where we send 307 is the part that Squid doesn't like, I
> > believe is because we are not sending the last chunk since the file is a
> > virus.
>
> Not exactly: Even if you send the last HTTP chunk, you would not be able
> to follow up with an HTTP 307 message. The HTTP fate of this transaction
> was sealed when you started trickling virgin HTTP message pieces back to
> Squid (and, hence, back to the HTTP client talking to Squid).
>
>
> HTH,
>
> Alex.
>
>
> > On Tue, Nov 26, 2019 at 4:52 PM Alex Rousskov wrote:
> >
> >     On 11/26/19 2:52 PM, Felipe Arturo Polanco wrote:
> >
> >      > We are sending an encapsulated HTTP 307 redirect webpage header
> >     whenever
> >      > a Virus is found and stop sending any other data after that
> >
> >     You must use ICAP status code 200 then. Make sure your encapsulated
> HTTP
> >     307 body (if any) is properly sent to Squid.
> >
> >
> >      > but squid
> >      > complains about ICAP failure when we do that:
> >      > Adaptation::Icap::Xaction::noteCommRead threw exception:
> >     corrupted chunk
> >      > size
> >
> >     What chunk size did Squid not like? You should be able to tell by
> >     looking at the packet capture of the failed transaction (or low-level
> >     Squid debugging).
> >
> >
> >      > We are not sending an ICAP header at this point because we
> >     already told
> >      > Squid ICAP 200 OK header and begun a body transaction, we send
> some
> >      > chunks back to the client for progress and hold the last part for
> >     scanning.
> >
> >     Are you sending HTTP 307 body chunks to Squid? How do you indicate
> that
> >     no more chunks will be coming?
> >
> >     It sounds like you are trying to cram two HTTP messages (one with the
> >     original HTTP response body prefix and one with a generated 307
> >     redirect) into one ICAP response, which is impossible, but perhaps I
> >     misunderstood your description. It would help if you post a sample
> (but
> >     complete) ICAP response that Squid does not like.
> >
> >
> >      > Ideally, we would like to just send our 307 to Squid and not
> >     having it
> >      > count as a failure.
> >
> >     Yes, a 200 ICAP response with an embedded HTTP 307 response should
> work
> >     just fine, but all its pieces should be properly formed (and there
> >     should be no extras).
> >
> >     Alex.
> >
> >
> >      > On Tue, Nov 26, 2019 at 3:44 PM Alex Rousskov wrote:
> >      >
> >      >     On 11/26/19 10:15 AM, Felipe Arturo Polanco wrote:
> >      >
> >      >     > While we can successfully scan our files and do content
> >     adaptation, we
> >      >     > have been struggling to find a way to close the ICAP
> >     transaction
> >      >     before
> >      >     > passing the whole body back to squid and at the same time
> >     avoid squid
> >      >     > marking one icap failure.
> >      >
> >      >     Squid needs a valid ICAP response. The right ICAP response
> >     status code
> >      >     depends on what you want Squid to do after receiving that
> >     response. You
> >      >     have mentioned what you do _not_ want Squid to do (i.e.
> >     increase the
> >      >     failure count), but that still leaves a lot of options.
> >      >
> >      >
> >      >     > This is for an ICAP server that does Virus scanning and if
> >     virus
> >      >     found,
> >      >     > the body is not sent back.
> >      >
> >      >     What do you want Squid to do when the ICAP service finds a
> >     virus? For
> >      >     example, what message do you want Squid to send to the next
> >     HTTP hop?
> >      >
> >      >     Alex.
> >      >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20191127/bc0dfa4b/attachment.html>