[squid-users] Squid full request logging
rousskov at measurement-factory.com
Thu Mar 4 21:53:23 UTC 2021
On 3/4/21 4:33 PM, Niels Hofmans wrote:
> Although I am not proficient in C for writing
> an ecap service, is there some binding available online for Go?
eCAP is C++ and, unfortunately, the first version of its API evidently
requires elevated programming skills. I am not aware of Go bindings, and
I would not recommend eCAP for those who are not fluent in C++ (or
cannot hire somebody fluent in C++).
> This was the reason I originally opted for an ICAP service since I can
> abstract Go behind the HTTP ICAP layer.
I do not know what "abstract Go behind the HTTP ICAP layer" means, but I
hope that you are not writing an ICAP service from scratch. If you do,
be prepared to battle many problems, both short- and long-term. ICAP
looks simple but it is difficult to get right.
> Now I understand this has its limitations, but AFAIK a preview cap at
> 100kb would be sufficient per request.
I am not sure Squid supports 100KB previews -- the internal buffers may
not be big enough for that. 64KB might be the limit, but do not quote me
BTW, please note that, AFAICT, Squid ignores icap_preview_size in
squid.conf (a bug) -- your ICAP service has to request the desired
maximum preview size via an ICAP OPTIONS response.
> Would you think that ... using two ICAP services.
> One would validate the headers and return OK or NOT (bypass=0), while
> the other only pushes the 1kb request/response to a queue.
> Ideally those two would be contacted simultaneously while only the first
> one is blocking.
> ..just thinking aloud tough.
Sorry, I cannot evaluate this design because I do not know what you want
to optimize and what your logging requirements/limitations are.
> On 4 Mar 2021, at 22:23, Alex Rousskov wrote:
> On 3/4/21 2:52 PM, Niels Hofmans wrote:
>> is it possible to do full request/response logging?
> Squid can log HTTP headers with %>h and %<h logformat codes.
> Squid cannot log HTTP message bodies.
>> I do not see the appropriate log_format directive in the docs.
>> I was hoping not having to do this in my ICAP service since this slows
>> down approval of the HTTP request. (Empty preview v.s. a request capped
>> at 1MB that needs to be sent over every time)
> FWIW, an ICAP or eCAP service can start responding to the request
> _before_ the service receives the entire HTTP message body. To get
> things going, all the service needs is HTTP headers (and even that is,
> technically, optional in some cases).
> Using an adaptation service is still an overhead, of course, but, very
> few legitimate Squid use cases involve logging message bodies, so there
> is no built-in mechanism optimized for that specific rare purpose
> (yet?). The fastest option available today is probably a dedicated eCAP
> service that refuses to adapt the message bit continues to receive (and
> log) the message body.
> squid-users mailing list
> squid-users at lists.squid-cache.org <mailto:squid-users at lists.squid-cache.org>
More information about the squid-users