[squid-users] logfileHandleWrite: daemon:/var/log/squid/access.log: error writing ((32) Broken pipe)

Tue Sep 6 22:02:53 UTC 2022

Hey Alex,

Thank you for your reply.
> Your syslog or cache.log might
> contain more info. Analyzing the timing/schedule of these problems may
> also be helpful in identifying the trigger.

After following this advice, I could easily see that this happens due to disk space running out on my server,
it seems that the logs has filled over 100GB of log data, since I made a configuration mistake (I think?) by setting this:
> quote_type
> logfile_rotate 0
>
If I remember and read correctly, this means that the rotation of the files is disabled and they will just keeping increasing
in size if left unchecked.

I have now gone ahead and changed all the configuration file to this setting:
> quote_type
> logfile_rotate 1
>
So now it should rotate once daily, and on the next rotation it should be deleted, and this is all handled by logrotate on Debian-based machines?

This is my / cat /etc/logrotate.d/squid:
➜ / cat /etc/logrotate.d/squid
#
# Logrotate fragment for squid.
#
/var/log/squid/*.log {
 daily
 compress
 delaycompress
 rotate 2
 missingok
 nocreate
 sharedscripts
 prerotate
 test ! -x /usr/sbin/sarg-reports || /usr/sbin/sarg-reports daily
 endscript
 postrotate
 test ! -e /var/run/squid.pid || test ! -x /usr/sbin/squid || /usr/sbin/squid -k rotate
 endscript
}

Is there a way for me to set it so it just get deleted every 24 or 12 hours without the archive first?

Thanks,
Roee
On 6 Sep 2022, 16:28 +0300, Alex Rousskov <rousskov at measurement-factory.com>, wrote:
> On 9/6/22 07:41, roee klinger wrote:
>
> > It is also important to know that I am running multiple Squid instances
> > on the same machine, they are all getting the error at the same time
>
> What external event(s) happen at that time? Something is probably
> sending a signal to the logging daemon process. It would be good to know
> what that something (and that signal) is. Your syslog or cache.log might
> contain more info. Analyzing the timing/schedule of these problems may
> also be helpful in identifying the trigger.
>
>
> > Is a possible workaround that might be just replacing the line with
> > this?
>
> > access_log /var/log/squid/access2.log
>
> As you know, this configuration (in this deprecated spelling or with and
> explicit "stdio:" prefix) will result in Squid workers writing to the
> log file directly instead of asking the logging daemon. This will,
> naturally, get rid of the pipe between workers and their daemons, and
> the associated broken pipe error.
>
> > or will this cause a problem?
>
> Impossible to say for sure without knowing whether your workers benefit
> from the anticipated performance advantages of avoiding blocking file
> I/O _and_ whether those advantages are real (in your environment). Too
> many variables and too many unknowns. I would treat this as an important
> (and potentially disruptive) configuration change and carefully test the
> outcome.
>
>
> HTH,
>
> Alex.
>
>
> > INFO -
> > Versions:
> >
> > Squid Cache: Version 4.10
> > Ubuntu 20.04.4 LTS
> >
> >
> > Example squid.conf:
> >
> > visible_hostname squid2
> >
> > access_log daemon:/var/log/squid/access2.log squid
> >
> > cache_log /var/log/squid/cache2.log
> >
> > pid_filename /var/run/squid2.pid
> >
> >
> > acl localnet src 0.0.0.1-0.255.255.255# RFC 1122 "this" network (LAN)
> >
> > acl localnet src 10.0.0.0/8 <http://10.0.0.0/8> # RFC 1918 local
> > private network (LAN)
> >
> > acl localnet src 100.64.0.0/10 <http://100.64.0.0/10># RFC 6598
> > shared address space (CGN)
> >
> > acl localnet src 169.254.0.0/16 <http://169.254.0.0/16> # RFC 3927
> > link-local (directly plugged) machines
> >
> > acl localnet src 172.16.0.0/12 <http://172.16.0.0/12># RFC 1918
> > local private network (LAN)
> >
> > acl localnet src 192.168.0.0/16 <http://192.168.0.0/16> # RFC 1918
> > local private network (LAN)
> >
> > acl localnet src fc00::/7 # RFC 4193 local private network range
> >
> > acl localnet src fe80::/10# RFC 4291 link-local (directly plugged)
> > machines
> >
> > acl SSL_ports port 443
> >
> > acl Safe_ports port 80# http
> >
> > acl Safe_ports port 21# ftp
> >
> > acl Safe_ports port 443 # https
> >
> > acl Safe_ports port 70# gopher
> >
> > acl Safe_ports port 210 # wais
> >
> > acl Safe_ports port 1025-65535# unregistered ports
> >
> > acl Safe_ports port 280 # http-mgmt
> >
> > acl Safe_ports port 488 # gss-http
> >
> > acl Safe_ports port 591 # filemaker
> >
> > acl Safe_ports port 777 # multiling http
> >
> > acl CONNECT method CONNECT
> >
> > http_access deny !Safe_ports
> >
> > http_access deny CONNECT !SSL_ports
> >
> > http_access allow localhost manager
> >
> > http_access deny manager
> >
> > # include /etc/squid/conf.d/*
> >
> > http_access allow localhost
> >
> > acl aws src *censored*
> >
> > http_access allow aws
> >
> > # http_access deny all
> >
> > tcp_outgoing_address *censored*
> >
> > http_port 10002
> >
> > coredump_dir /var/spool/squid
> >
> > refresh_pattern ^ftp: 144020% 10080
> >
> > refresh_pattern ^gopher:14400%1440
> >
> > refresh_pattern -i (/cgi-bin/|\?) 0 0%0
> >
> > refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0
> > refresh-ims
> >
> > refresh_pattern \/Release(|\.gpg)$ 0 0% 0 refresh-ims
> >
> > refresh_pattern \/InRelease$ 0 0% 0 refresh-ims
> >
> > refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims
> >
> > refresh_pattern . 0 20% 4320
> >
> >
> > shutdown_lifetime 1 seconds
> >
> > logfile_rotate 0
> >
> > max_filedescriptors 16384
> >
> > dns_nameservers 8.8.8.8 8.8.4.4 1.1.1.1
> >
> > cache deny all
> >
> > cache_dir null /tmp
> >
> > via off
> >
> > forwarded_for off
> >
> > request_header_access From deny all
> >
> > request_header_access Server deny all
> >
> > request_header_access WWW-Authenticate deny all
> >
> > request_header_access Link deny all
> >
> > request_header_access Cache-Control deny all
> >
> > request_header_access Proxy-Connection deny all
> >
> > request_header_access X-Cache deny all
> >
> > request_header_access X-Cache-Lookup deny all
> >
> > request_header_access Via deny all
> >
> > request_header_access X-Forwarded-For deny all
> >
> > request_header_access Pragma deny all
> >
> > request_header_access Keep-Alive deny all
> >
> > dns_v4_first on
> >
> >
> > Example service file:
> >
> > ## Copyright (C) 1996-2020 The Squid Software Foundation and
> > contributors
> >
> > ##
> >
> > ## Squid software is distributed under GPLv2+ license and includes
> >
> > ## contributions from numerous individuals and organizations.
> >
> > ## Please see the COPYING and CONTRIBUTORS files for details.
> >
> > ##
> >
> >
> > [Unit]
> >
> > Description=Squid Web Proxy Server
> >
> > Documentation=man:squid(8)
> >
> > After=network.target network-online.target nss-lookup.target
> >
> >
> > [Service]
> >
> > Type=forking
> >
> > PIDFile=/var/run/squid2.pid
> >
> > ExecStartPre=/usr/sbin/squid --foreground -z -f /etc/squid/squid2.conf
> >
> > ExecStart=/usr/sbin/squid -sYC -f /etc/squid/squid2.conf
> >
> > ExecReload=/bin/kill -HUP $MAINPID
> >
> > KillMode=mixed
> >
> >
> > [Install]
> >
> > WantedBy=multi-user.target
> >
> >
> >
> > Permissions:
> >
> > ➜  ls -alt /etc/squid/
> > total 128
> > drwxr-xr-x   2 root root 4096 Sep  6 11:33 .
> > -rw-r--r--   1 root root 2831 Sep  6 11:33 squid7.conf
> > drwxr-xr-x 116 root root 4096 Sep  6 11:33 ..
> > -rw-r--r--   1 root root 2830 Sep  6 11:33 squid2.conf
> > -rw-r--r--   1 root root 2836 Sep  6 11:33 squid13.conf
> > -rw-r--r--   1 root root 2836 Sep  6 11:32 squid23.conf
> > -rw-r--r--   1 root root 2836 Sep  6 11:32 squid19.conf
> > -rw-r--r--   1 root root 2832 Sep  6 11:32 squid1.conf
> > -rw-r--r--   1 root root 2836 Sep  6 11:32 squid17.conf
> > -rw-r--r--   1 root root 2832 Sep  6 11:31 squid4.conf
> > -rw-r--r--   1 root root 2834 Sep  6 11:31 squid21.conf
> > -rw-r--r--   1 root root 2833 Sep  6 11:31 squid25.conf
> > -rw-r--r--   1 root root 2834 Sep  6 11:31 squid12.conf
> > -rw-r--r--   1 root root 2832 Sep  6 11:31 squid3.conf
> > -rw-r--r--   1 root root 2836 Sep  6 11:30 squid10.conf
> > -rw-r--r--   1 root root 2835 Sep  6 11:30 squid11.conf
> > -rw-r--r--   1 root root 2833 Sep  6 11:30 squid18.conf
> > -rw-r--r--   1 root root 2830 Sep  6 11:30 squid8.conf
> > -rw-r--r--   1 root root 2830 Sep  6 11:30 squid6.conf
> > -rw-r--r--   1 root root 2833 Sep  6 11:30 squid28.conf
> > -rw-r--r--   1 root root 2830 Sep  6 11:25 squid9.conf
> > -rw-r--r--   1 root root 2836 Sep  6 11:25 squid24.conf
> > -rw-r--r--   1 root root 2835 Sep  6 11:25 squid22.conf
> > -rw-r--r--   1 root root 2837 Sep  6 11:25 squid20.conf
> > -rw-r--r--   1 root root 2836 Sep  6 11:25 squid16.conf
> > -rw-r--r--   1 root root 2835 Sep  6 11:25 squid15.conf
> > -rw-r--r--   1 root root 2836 Sep  6 11:25 squid14.conf
> > -rw-r--r--   1 root root 2831 Sep  6 11:25 squid5.conf
> > -rw-r--r--   1 root root 2833 Sep  6 11:25 squid27.conf
> > -rw-r--r--   1 root root 2835 Sep  6 11:25 squid26.conf
> > -rw-r--r--   1 root root 2835 Sep  6 11:25 squid30.conf
> > -rw-r--r--   1 root root 2835 Sep  6 11:25 squid29.conf
> >
> >
> > _______________________________________________
> > squid-users mailing list
> > squid-users at lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20220907/a07dbea2/attachment-0001.htm>