[squid-users] Do squid logs performence affect general request performece?

[Alex Rousskov]

On 9/9/22 08:49, roee klinger wrote:
> Hello,
> 
> I have just recently started exploring ingesting logs from Squid via 
> Logstash (TCP log ingestion)
> 
> I now have to make the decision of how to deploy Logstash, it seems to 
> me that I have two options:
> 
>  1. Deploy a Logstash instance for every region where I have a Squid
>     instance running, resulting in the lowest latency possible between
>     Squid and Logstash.
>  2. Deploy a central Logstash instance for all Squid instance worldwide,
>     which is the cheapest and easiest option, but will have high latency
>     for some of the Squid instance which are located far away
>     geographically from the Logstash instance.
> 
> I already know that if the log server crashes in Squid, then Squid will 
> crash with a fatal error,

FYI: Logging error handling is configurable via "access_log on-error".


> so Squid takes logging very seriously, so my 
> question is: does TCP logging performance in Squid effect the total 
> request performance, or are they independent from each other?

TCP logger uses asynchronous non-blocking I/O. Individual HTTP 
transactions do not wait for individual log records to be sent. Needless 
to say, logging does consume resources and, hence, may affect overall 
HTTP transaction performance. And if the log record recipient is too 
slow, then Squid will die and/or log records will be dropped.


> What would be you recommendations, if request performance is crucial 
> (and log performance is not)?

I would not be surprised if you would not be able to measure meaningful 
performance difference among these modules. From general performance 
point of view, and module-specific bugs and limitations notwithstanding, 
I would expect "daemon" and "udp" modules to have smaller performance 
overhead/cost for Squid workers than the "tcp" module because I expect 
that TCP has to "do more" than stdout I/O and UDP. However, I would not 
base this decision on such high-level speculations and test various 
options instead.


HTH,

Alex.