[squid-users] Squid full request logging

[NgTech LTD]

Hey Niels,

Take a peek at:
https://github.com/andybalholm/redwood

I am using it in production and it was written because of squid limitations.
squid is great but take a peek and see how it works for you.
I have 2 servers in ha cluster which works great.

An example I wrote to filter youtube traffic is at:
https://github.com/elico/yt-classification-service-example

let me know if it helps you or gives you any direction.

בתאריך יום ה׳, 4 במרץ 2021, 23:33, מאת Niels Hofmans ‏<hello at ironpeak.be>:

> Hi Alex,
>
> Thanks for the feedback. Although I am not proficient in C for writing an
> ecap service, is there some binding available online for Go?
> This was the reason I originally opted for an ICAP service since I can
> abstract Go behind the HTTP ICAP layer.
> Now I understand this has its limitations, but AFAIK a preview cap at
> 100kb would be sufficient per request.
> But this will slow down my current setup greatly, as I’m currently sending
> -only- the headers.
>
> Would you think that a) using Go for the ecap adapter or b) using two ICAP
> services.
> One would validate the headers and return OK or NOT (bypass=0), while the
> other only pushes the 1kb request/response to a queue.
> Ideally those two would be contacted simultaneously while only the first
> one is blocking.
> ..just thinking aloud tough.
>
> Regards,
> Niels Hofmans
>
> SITE   https://ironpeak.be
> BTW   BE0694785660
> BANK BE76068909740795
>
> On 4 Mar 2021, at 22:23, Alex Rousskov <rousskov at measurement-factory.com>
> wrote:
>
> On 3/4/21 2:52 PM, Niels Hofmans wrote:
>
> is it possible to do full request/response logging?
>
>
> Squid can log HTTP headers with %>h and %<h logformat codes.
>
> Squid cannot log HTTP message bodies.
>
>
> I do not see the appropriate log_format directive in the docs.
> I was hoping not having to do this in my ICAP service since this slows
> down approval of the HTTP request. (Empty preview v.s. a request capped
> at 1MB that needs to be sent over every time)
>
>
> FWIW, an ICAP or eCAP service can start responding to the request
> _before_ the service receives the entire HTTP message body. To get
> things going, all the service needs is HTTP headers (and even that is,
> technically, optional in some cases).
>
> Using an adaptation service is still an overhead, of course, but, very
> few legitimate Squid use cases involve logging message bodies, so there
> is no built-in mechanism optimized for that specific rare purpose
> (yet?). The fastest option available today is probably a dedicated eCAP
> service that refuses to adapt the message bit continues to receive (and
> log) the message body.
>
>
> HTH,
>
> Alex.
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210305/a26f31c4/attachment.htm>