[squid-users] Icap preview size

[Niels Hofmans]

On 6 Mar 2021, at 21:33, Niels Hofmans <hello at ironpeak.be> wrote:

Hi Alex,

I fixed a bug in the go-icap/icap library, seen here:
But this made me wonder about the following: https://i.imgur.com/bA3w8YT.png <https://i.imgur.com/bA3w8YT.png>

Here you can see squid issuing a RESPMOD to my ICAP server with a Preview of “0”, but it does contain a HTTP response.
And since my ICAP service will not return a HTTP response since the Preview size is 0, my http client receives nothing from squid.
I checked but I don’t think I see a 100 continuation from ICAP in the dump anymore.

Tcpdump: https://www.mediafire.com/file/sr7hvk9ezepoc8d/icap.pcap3/file <https://www.mediafire.com/file/sr7hvk9ezepoc8d/icap.pcap3/file>
(I’ve revoked my session cookies for this dump)

One step at a time closer to the truth! :-)

Regards,
Niels Hofmans

SITE   https://ironpeak.be <https://ironpeak.be/>
BTW   BE0694785660
BANK BE76068909740795

On 6 Mar 2021, at 19:29, Alex Rousskov <rousskov at measurement-factory.com <mailto:rousskov at measurement-factory.com>> wrote:

On 3/6/21 5:09 AM, Niels Hofmans wrote:

> I’ve uploaded a tcpdump sample of where I’m requesting an image through
> squid + icap and where you can see the image response being posted fully
> to the ICAP service.

AFAICT, all ICAP REQMOD and RESPMOD requests in your packet capture send
zero-size Preview containing just the HTTP headers (e.g., see frame
#44). In one case, Squid also sends the HTTP response body but only
because your ICAP service explicitly requests that HTTP response body by
responding to Squid Preview with ICAP 100 Continue control message
(e.g., see frame #48)!

If your ICAP service does not want to see an HTTP body, then it should
not ask for it. It should respond (usually with ICAP 200 or ICAP 204)
based on the Preview alone, without sending ICAP 100 Continue control
message first.


HTH,

Alex.


> On 5 Mar 2021, at 23:32, Alex Rousskov wrote:
> 
> On 3/5/21 5:21 PM, Niels Hofmans wrote:
> 
>> I receive that large payload right after an OPTIONS call to my ICAP
>> endpoint. It is the preview.
> 
> The timing of an ICAP request does not determine whether that request
> has a Preview stage and whether the "unexpected" body bytes were sent
> during that Preview stage. I am asking these questions because I want to
> determine whether Squid increases Preview size beyond what your server
> has requested or does not send Preview at all.
> 
> Please share the ICAP headers of the problematic REQMOD request and the
> last chunk metadata of that request body. You can get the latter from a
> packet capture if your ICAP server does not report it in a convenient
> form. In fact, sharing (a pointer to) the packet capture of the whole
> problematic ICAP request is probably a good idea!
> 
> Alex.
> 
> 
> 
>> On 5 Mar 2021, at 17:21, Alex Rousskov wrote:
>> 
>> On 3/5/21 2:55 AM, Niels Hofmans wrote:
>> 
>>> One more: I believe ICAP is not respecting the Preview header for REQMOD
>>> nor RESPMOD.
>> 
>>> For the REQMOD OPTIONS requests, I respond with:
>>> 
>>> ICAP/1.0 200 OK
>>> Allow: 200,204
>>> Connection: close
>>> Date: Fri, 05 Mar 2021 07:34:56 GMT
>>> Encapsulated: null-body=0
>>> Methods: REQMOD
>>> Preview: 64000
>>> 
>>> However in my ICAP service, I often receive a request body over 64000
>>> bytes, e.g. 1000000 request bytes.
>> 
>> Receive when? During preview, instead of Preview, and/or after Preview?
>> 
>> 
>>> I also noticed that a "Preview: 0” will not have any effect and the ICAP
>>> service will still receive a complete HTTP REQMOD+REQRESP body.
>> 
>> Receive when? During preview, instead of Preview, and/or after Preview?
>> 
>> Alex.
>> 
>> 
>> 
>>> icap_enable on
>>> icap_connect_timeout 1 second
>>> icap_service_failure_limit -1
>>> icap_send_client_ip    on
>>> icap_preview_size 0
>>> icap_persistent_connections on
>>> icap_service service_req reqmod_precache bypass=0
>>> icap://172.17.0.2:1344/request <icap://172.17.0.2:1344/request> <icap://172.17.0.2:1344/request <icap://172.17.0.2:1344/request>>
>>> <icap://172.17.0.2:1344/request <icap://172.17.0.2:1344/request> <icap://172.17.0.2:1344/request <icap://172.17.0.2:1344/request>>>
>>> icap_service service_res respmod_precache bypass=0
>>> icap://172.17.0.2:1344/response <icap://172.17.0.2:1344/response> <icap://172.17.0.2:1344/response <icap://172.17.0.2:1344/response>>
>>> <icap://172.17.0.2:1344/response <icap://172.17.0.2:1344/response> <icap://172.17.0.2:1344/response <icap://172.17.0.2:1344/response>>>
>>> icap_preview_enable on
>>> adaptation_access service_req allow all
>>> adaptation_access service_res allow all

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210306/474387b3/attachment-0001.htm>