[squid-users] Adding headers in ICAP server with no preview

[Moti Berger]

Hi

If the ICAP server sets 'Preview: 0' in the OPTIONS it means that when the
ICAP client sends a request, it should not contain the body.
This is the REQMOD request:

> F..n...DREQMOD icap://censor-req.proxy:14590/request ICAP/1.0
> Host: censor-req.proxy:14590
> Date: Mon, 18 Jan 2021 11:34:54 GMT
> Encapsulated: req-hdr=0, req-body=222
> Preview: 0
> Allow: 204, trailers
> X-custom-header: data
>
> POST http://www.dst-server.com:22222/v1/test HTTP/1.1
> User-Agent: python-requests/2.25.1
> Accept-Encoding: gzip, deflate
> Accept: */*
> Content-Length: 10
> Content-Type: application/json
> Host: www.dst-server.com:22222
>

The ICAP 'Encapsulated' header has a req-body even though no 'body' should
be in this request.
I wonder why in this case the 'Encapsulated' header doesn't contain
null-body.
I could not find any reference to this case in the RFC3507.
The ICAP server has no way to encapsulate the HTTP request body if it
didn't get it.

I want to avoid sending the body because the adaptation is body agnostic.


On Sun, Jan 17, 2021 at 11:34 PM Alex Rousskov <
rousskov at measurement-factory.com> wrote:

> On 1/17/21 3:08 PM, Moti Berger wrote:
> > What should the ICAP response look like?
>
> The vast majority off ICAP responses containing an HTTP POST message
> will look like ICAP header + HTTP header + HTTP body. Please see RFC
> 3507 and its errata for examples of and discussion about those three
> components. It should help avoid guessing and developing by examples
> (which usually leads to bugs, especially where ICAP is involved).
>
>
> > What I do is to reply like this:
> >
> >     (dI./M..ICAP/1.0 200 OK
> >     ISTag: "SjIzlRA4te41axxcDOoiSl6rBRg4ZK"
> >     Date: Sun, 17 Jan 2021 19:34:12 GMT
> >     Server: BaseICAP/1.0 Python/3.6.12
> >     Encapsulated: req-hdr=0, req-body=360
> >
> >     POST http://www.dst-server.com:22222/v1/test HTTP/1.1
> >     x-new-header: {"key": "value"}
> >     user-agent: python-requests/2.25.1
> >     accept-encoding: gzip, deflate
> >     accept: */*
> >     content-length: 16
> >     content-type: application/json
> >     host: www.dst-server.com:22222 <http://www.dst-server.com:22222>
>
>
> FYI: The above incomplete ICAP response promises an HTTP request body,
> both on the ICAP level (req-body) and on the HTTP level (content-length:
> 16).
>
>
> > As I said, I use 'Preview: 0' since I don't mind the body. The question
> > is whether declaring the body starts at X (req-body=X) is OK even though
> > I don't have a body to send?
>
> It is not OK not to send the body. Encapsulated:req-body does more than
> declaring where the encapsulated headers end. It also promises an
> embedded HTTP body after those headers. You must encapsulate the body if
> the HTTP message should have one. You cannot adapt the header of an HTTP
> message with a body without also sending the HTTP body (virgin or adapted).
>
> Preview is pretty much irrelevant in this context -- the ICAP protocol
> does not care how the ICAP service gets the HTTP body to include in the
> ICAP response.
>
> There are unofficial ICAP extensions that make it possible to tell the
> ICAP client to reuse the body it has buffered while adapting the header,
> but you should get the baseline case working before bothering with those
> extensions -- they are optimizations that are not applicable to some
> transactions.
>
>
> > I think having req-null=X is bad since it
> > probably tells squid that I decided the adapted request should have no
> > body, but that's only a guess.
>
> If you meant to say "null-body", then you guessed correctly -- null-body
> means the adapted HTTP message has no body. That is not what you want to
> say when adapting most HTTP POST messages.
>
>
> HTH,
>
> Alex.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210118/1a63b352/attachment.htm>