[squid-users] What is the proper way to close an ICAP transaction?
Felipe Arturo Polanco
felipeapolanco at gmail.com
Wed Nov 27 18:42:10 UTC 2019
Can you describe the process of option 2?
Terminate Squid-to-client transmission while indicating (to that same
client) that the HTTP response body was cut short (i.e. that the
delivery of the response was aborted).
What TCP Flag or ICAP header should the ICAP server send in order to inform
connection aborted??
On Wed, Nov 27, 2019 at 12:44 PM Alex Rousskov <
rousskov at measurement-factory.com> wrote:
> On 11/27/19 11:01 AM, Felipe Arturo Polanco wrote:
>
> > How can we then terminate an ICAP 200 OK transaction to squid without
> > sending the complete body back to it? We don't want squid to mark an
> > ICAP failure on us if we just close the TCP connection.
>
> To properly answer your question, I have to come back to the question I
> asked earlier: What do you want Squid to do when your ICAP service finds
> a virus after trickling a few virgin HTTP body bytes to the HTTP client?
>
> The "we want Squid to send an HTTP 307 redirect to the client" desire
> you responded with earlier was ruled impossible to satisfy in your
> trickling environment. A few realistic options remain though, including:
>
> 1. Terminate Squid-to-client transmission as if the whole virgin HTTP
> response body was sent to the client.
>
> 2. Terminate Squid-to-client transmission while indicating (to that same
> client) that the HTTP response body was cut short (i.e. that the
> delivery of the response was aborted).
>
> 3. #2 plus annotate the transaction specially in Squid access.log and/or
> detect this special outcome using Squid ACLs.
>
> Alex.
>
>
> > On Wed, Nov 27, 2019 at 10:54 AM Alex Rousskov wrote:
> >
> > On 11/26/19 4:12 PM, Felipe Arturo Polanco wrote:
> >
> > > The flow is the following:
> > > ICAP transaction is sent to ICAP server with a PREVIEW header
> > > ICAP server sends ICAP header 100 Continue
> > > ICAP server sends ICAP header 200 OK to start data transfer
> >
> > (*) Your notes below imply that the ICAP server also sends the
> embedded
> > HTTP message header back to Squid (and not just the ICAP 200 header).
> >
> >
> > > <data transfer begins>
> > > ICAP server receives a chunk, checks if its the last chunk, if not
> > then
> > > append to temp file and send it back to Squid;
> >
> > The "send it back to Squid" part implies that not only your ICAP
> server
> > sent an ICAP 200 response header, but it sent the HTTP message
> header as
> > well. See (*) above. Once the ICAP server sent the HTTP message
> header,
> > it is committed to finish sending that HTTP message (and nothing
> else).
> >
> >
> > > if it is the last chunk then analyze the temp file for virus.
> > > <repeat for next data transfer>
> > > If virus found then send encapsulated HTTP header 307 redirect.
> >
> > This is an ICAP service bug: One cannot send a second encapsulated
> HTTP
> > message in one ICAP response. A single ICAP response cannot contain
> > multiple HTTP messages. The ICAP protocol does not allow for that,
> and
> > there would be no way to actually support something like that in an
> ICAP
> > client because the HTTP message cannot be interrupted and replaced
> with
> > another mid-flight. You may assume that the HTTP client is already
> > seeing the beginning of the first HTTP response. The train has left
> the
> > station.
> >
> > If the above is accurate, and you are using the ICAP service
> correctly,
> > then the ICAP service that allowed you to do the above is badly
> broken,
> > probably written by somebody who thought that ICAP is "easy". You
> may be
> > better off reusing an existing higher-quality ICAP service instead.
> >
> >
> > > If virus not found, send the last chunk to squid.
> > >
> > > The part where we send 307 is the part that Squid doesn't like, I
> > > believe is because we are not sending the last chunk since the
> > file is a
> > > virus.
> >
> > Not exactly: Even if you send the last HTTP chunk, you would not be
> able
> > to follow up with an HTTP 307 message. The HTTP fate of this
> transaction
> > was sealed when you started trickling virgin HTTP message pieces
> back to
> > Squid (and, hence, back to the HTTP client talking to Squid).
> >
> >
> > HTH,
> >
> > Alex.
> >
> >
> > > On Tue, Nov 26, 2019 at 4:52 PM Alex Rousskov wrote:
> > >
> > > On 11/26/19 2:52 PM, Felipe Arturo Polanco wrote:
> > >
> > > > We are sending an encapsulated HTTP 307 redirect webpage
> header
> > > whenever
> > > > a Virus is found and stop sending any other data after that
> > >
> > > You must use ICAP status code 200 then. Make sure your
> > encapsulated HTTP
> > > 307 body (if any) is properly sent to Squid.
> > >
> > >
> > > > but squid
> > > > complains about ICAP failure when we do that:
> > > > Adaptation::Icap::Xaction::noteCommRead threw exception:
> > > corrupted chunk
> > > > size
> > >
> > > What chunk size did Squid not like? You should be able to tell
> by
> > > looking at the packet capture of the failed transaction (or
> > low-level
> > > Squid debugging).
> > >
> > >
> > > > We are not sending an ICAP header at this point because we
> > > already told
> > > > Squid ICAP 200 OK header and begun a body transaction, we
> > send some
> > > > chunks back to the client for progress and hold the last
> > part for
> > > scanning.
> > >
> > > Are you sending HTTP 307 body chunks to Squid? How do you
> > indicate that
> > > no more chunks will be coming?
> > >
> > > It sounds like you are trying to cram two HTTP messages (one
> > with the
> > > original HTTP response body prefix and one with a generated 307
> > > redirect) into one ICAP response, which is impossible, but
> > perhaps I
> > > misunderstood your description. It would help if you post a
> > sample (but
> > > complete) ICAP response that Squid does not like.
> > >
> > >
> > > > Ideally, we would like to just send our 307 to Squid and not
> > > having it
> > > > count as a failure.
> > >
> > > Yes, a 200 ICAP response with an embedded HTTP 307 response
> > should work
> > > just fine, but all its pieces should be properly formed (and
> there
> > > should be no extras).
> > >
> > > Alex.
> > >
> > >
> > > > On Tue, Nov 26, 2019 at 3:44 PM Alex Rousskov wrote:
> > > >
> > > > On 11/26/19 10:15 AM, Felipe Arturo Polanco wrote:
> > > >
> > > > > While we can successfully scan our files and do
> content
> > > adaptation, we
> > > > > have been struggling to find a way to close the ICAP
> > > transaction
> > > > before
> > > > > passing the whole body back to squid and at the same
> time
> > > avoid squid
> > > > > marking one icap failure.
> > > >
> > > > Squid needs a valid ICAP response. The right ICAP
> response
> > > status code
> > > > depends on what you want Squid to do after receiving
> that
> > > response. You
> > > > have mentioned what you do _not_ want Squid to do (i.e.
> > > increase the
> > > > failure count), but that still leaves a lot of options.
> > > >
> > > >
> > > > > This is for an ICAP server that does Virus scanning
> > and if
> > > virus
> > > > found,
> > > > > the body is not sent back.
> > > >
> > > > What do you want Squid to do when the ICAP service
> finds a
> > > virus? For
> > > > example, what message do you want Squid to send to the
> next
> > > HTTP hop?
> > > >
> > > > Alex.
> > > >
> > >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20191127/730143d2/attachment.html>
More information about the squid-users
mailing list