<div dir="auto"><div>Good one, Alex.<div dir="auto"><br></div><div dir="auto">For this specific use case you need a special rotate script which will know the confs file and will loop over them.</div><div dir="auto">Later on I will try to see if yave one of these on my servers.</div><div dir="auto">Basically you will need an array of config files and loop on them.</div><div dir="auto"><br></div><div dir="auto">The pid shouldn't be relevevant for a rotate operation but it depends on the nature of the system.(on a 24/7 system you should know about a service that is down way before the logrotate happpens)</div><div dir="auto">If you have a set of config files you can generate a set of postrotate commands compared to a special script.</div><div dir="auto"><br></div>Let me know if this solution might fit for your use case.</div><div dir="auto"><br></div><div dir="auto">Eliezer<br><br><div class="gmail_quote" dir="auto"><div dir="ltr" class="gmail_attr">בתאריך יום ד׳, 7 בספט׳ 2022, 3:53, מאת Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com">rousskov@measurement-factory.com</a>>:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> > pid_filename /var/run/squid2.pid<br>
<br>
> postrotate<br>
> test ! -e /var/run/squid.pid || ... /usr/sbin/squid -k rotate<br>
> endscript<br>
<br>
I spotted one more (potentially critical) problem: Your Squid <br>
configuration sets pid_filename to /var/run/squid2.pid but your <br>
logrotate configuration assumes Squid uses /var/run/squid.pid.<br>
<br>
IMHO, in general, it is best not to guess where Squid has its PID if you <br>
are using "squid -k ...". If you want to test whether Squid is currently <br>
running, try using "squid -k check" instead.<br>
<br>
<br>
HTH,<br>
<br>
Alex.<br>
<br>
<br>
<br>
On 9/6/22 20:45, Alex Rousskov wrote:<br>
> On 9/6/22 18:02, roee klinger wrote:<br>
>> it seems that the logs has filled over 100GB of log data, since I made <br>
>> a configuration mistake (I think?) by setting this:<br>
>><br>
>> logfile_rotate 0<br>
> <br>
> This is correct setting when using an external log rotation tool like <br>
> the logrotate daemon. More on that below.<br>
> <br>
> <br>
>> If I remember and read correctly, this means that the rotation of the <br>
>> files is disabled and they will just keeping increasing<br>
>> in size if left unchecked.<br>
> <br>
> To be more precise, this means that you are relying on an external tool <br>
> to rename the log files. With this setting, Squid rotate command closes <br>
> the access log and opens a new one (under the same name). While that <br>
> might sound useless, it is the right (and necessary) thing for Squid to <br>
> do when combined with the correct external log rotation setup.<br>
> <br>
> <br>
>> I have now gone ahead and changed all the configuration file to this <br>
>> setting:<br>
>><br>
>> logfile_rotate 1<br>
>><br>
>> So now it should rotate once daily, and on the next rotation it should <br>
>> be deleted, and this is all handled by logrotate on Debian-based <br>
>> machines?<br>
> <br>
> AFAIK, if you are using an external (to Squid) tool like logrotate, you <br>
> should be setting logfile_rotate to zero.<br>
> <br>
> <br>
>> This is my / cat /etc/logrotate.d/squid:<br>
>> ➜ / cat /etc/logrotate.d/squid<br>
>> #<br>
>> # Logrotate fragment for squid.<br>
>> #<br>
>> /var/log/squid/*.log {<br>
>> daily<br>
>> compress<br>
>> delaycompress<br>
>> rotate 2<br>
>> missingok<br>
>> nocreate<br>
>> sharedscripts<br>
>> prerotate<br>
>> test ! -x /usr/sbin/sarg-reports || /usr/sbin/sarg-reports daily<br>
>> endscript<br>
>> postrotate<br>
>> test ! -e /var/run/squid.pid || test ! -x /usr/sbin/squid || <br>
>> /usr/sbin/squid -k rotate<br>
>> endscript<br>
>> }<br>
> <br>
> This is not my area of expertise, but the above configuration does not <br>
> look 100% correct to me: sarg-reports execution failures should have no <br>
> effect on log rotation but does (AFAICT). There may be other problems <br>
> (e.g., I do not know whether your /usr/sbin/squid finds the right Squid <br>
> configuration file). I hope sysadmin experts on this mailing list will <br>
> help you polish this.<br>
> <br>
> You should be able to test whether the above is working (e.g., by asking <br>
> logrotate to rotate). Testing is critical even if you do end up getting <br>
> expert log rotation help on this list (this email is not it!).<br>
> <br>
> <br>
> HTH,<br>
> <br>
> Alex.<br>
> <br>
> <br>
>> Is there a way for me to set it so it just get deleted every 24 or 12 <br>
>> hours without the archive first?<br>
>><br>
>> Thanks,<br>
>> Roee<br>
>> On 6 Sep 2022, 16:28 +0300, Alex Rousskov <br>
>> <<a href="mailto:rousskov@measurement-factory.com" target="_blank" rel="noreferrer">rousskov@measurement-factory.com</a>>, wrote:<br>
>>> On 9/6/22 07:41, roee klinger wrote:<br>
>>><br>
>>>> It is also important to know that I am running multiple Squid instances<br>
>>>> on the same machine, they are all getting the error at the same time<br>
>>><br>
>>> What external event(s) happen at that time? Something is probably<br>
>>> sending a signal to the logging daemon process. It would be good to know<br>
>>> what that something (and that signal) is. Your syslog or cache.log might<br>
>>> contain more info. Analyzing the timing/schedule of these problems may<br>
>>> also be helpful in identifying the trigger.<br>
>>><br>
>>><br>
>>>> Is a possible workaround that might be just replacing the line with<br>
>>>> this?<br>
>>><br>
>>>> access_log /var/log/squid/access2.log<br>
>>><br>
>>> As you know, this configuration (in this deprecated spelling or with and<br>
>>> explicit "stdio:" prefix) will result in Squid workers writing to the<br>
>>> log file directly instead of asking the logging daemon. This will,<br>
>>> naturally, get rid of the pipe between workers and their daemons, and<br>
>>> the associated broken pipe error.<br>
>>><br>
>>>> or will this cause a problem?<br>
>>><br>
>>> Impossible to say for sure without knowing whether your workers benefit<br>
>>> from the anticipated performance advantages of avoiding blocking file<br>
>>> I/O _and_ whether those advantages are real (in your environment). Too<br>
>>> many variables and too many unknowns. I would treat this as an important<br>
>>> (and potentially disruptive) configuration change and carefully test the<br>
>>> outcome.<br>
>>><br>
>>><br>
>>> HTH,<br>
>>><br>
>>> Alex.<br>
>>><br>
>>><br>
>>>> INFO -<br>
>>>> Versions:<br>
>>>><br>
>>>> Squid Cache: Version 4.10<br>
>>>> Ubuntu 20.04.4 LTS<br>
>>>><br>
>>>><br>
>>>> Example squid.conf:<br>
>>>><br>
>>>> visible_hostname squid2<br>
>>>><br>
>>>> access_log daemon:/var/log/squid/access2.log squid<br>
>>>><br>
>>>> cache_log /var/log/squid/cache2.log<br>
>>>><br>
>>>> pid_filename /var/run/squid2.pid<br>
>>>><br>
>>>><br>
>>>> acl localnet src 0.0.0.1-0.255.255.255# RFC 1122 "this" network (LAN)<br>
>>>><br>
>>>> acl localnet src <a href="http://10.0.0.0/8" rel="noreferrer noreferrer" target="_blank">10.0.0.0/8</a> <<a href="http://10.0.0.0/8" rel="noreferrer noreferrer" target="_blank">http://10.0.0.0/8</a>> # RFC 1918 local<br>
>>>> private network (LAN)<br>
>>>><br>
>>>> acl localnet src <a href="http://100.64.0.0/10" rel="noreferrer noreferrer" target="_blank">100.64.0.0/10</a> <<a href="http://100.64.0.0/10" rel="noreferrer noreferrer" target="_blank">http://100.64.0.0/10</a>># RFC 6598<br>
>>>> shared address space (CGN)<br>
>>>><br>
>>>> acl localnet src <a href="http://169.254.0.0/16" rel="noreferrer noreferrer" target="_blank">169.254.0.0/16</a> <<a href="http://169.254.0.0/16" rel="noreferrer noreferrer" target="_blank">http://169.254.0.0/16</a>> # RFC 3927<br>
>>>> link-local (directly plugged) machines<br>
>>>><br>
>>>> acl localnet src <a href="http://172.16.0.0/12" rel="noreferrer noreferrer" target="_blank">172.16.0.0/12</a> <<a href="http://172.16.0.0/12" rel="noreferrer noreferrer" target="_blank">http://172.16.0.0/12</a>># RFC 1918<br>
>>>> local private network (LAN)<br>
>>>><br>
>>>> acl localnet src <a href="http://192.168.0.0/16" rel="noreferrer noreferrer" target="_blank">192.168.0.0/16</a> <<a href="http://192.168.0.0/16" rel="noreferrer noreferrer" target="_blank">http://192.168.0.0/16</a>> # RFC 1918<br>
>>>> local private network (LAN)<br>
>>>><br>
>>>> acl localnet src fc00::/7 # RFC 4193 local private network range<br>
>>>><br>
>>>> acl localnet src fe80::/10# RFC 4291 link-local (directly plugged)<br>
>>>> machines<br>
>>>><br>
>>>> acl SSL_ports port 443<br>
>>>><br>
>>>> acl Safe_ports port 80# http<br>
>>>><br>
>>>> acl Safe_ports port 21# ftp<br>
>>>><br>
>>>> acl Safe_ports port 443 # https<br>
>>>><br>
>>>> acl Safe_ports port 70# gopher<br>
>>>><br>
>>>> acl Safe_ports port 210 # wais<br>
>>>><br>
>>>> acl Safe_ports port 1025-65535# unregistered ports<br>
>>>><br>
>>>> acl Safe_ports port 280 # http-mgmt<br>
>>>><br>
>>>> acl Safe_ports port 488 # gss-http<br>
>>>><br>
>>>> acl Safe_ports port 591 # filemaker<br>
>>>><br>
>>>> acl Safe_ports port 777 # multiling http<br>
>>>><br>
>>>> acl CONNECT method CONNECT<br>
>>>><br>
>>>> http_access deny !Safe_ports<br>
>>>><br>
>>>> http_access deny CONNECT !SSL_ports<br>
>>>><br>
>>>> http_access allow localhost manager<br>
>>>><br>
>>>> http_access deny manager<br>
>>>><br>
>>>> # include /etc/squid/conf.d/*<br>
>>>><br>
>>>> http_access allow localhost<br>
>>>><br>
>>>> acl aws src *censored*<br>
>>>><br>
>>>> http_access allow aws<br>
>>>><br>
>>>> # http_access deny all<br>
>>>><br>
>>>> tcp_outgoing_address *censored*<br>
>>>><br>
>>>> http_port 10002<br>
>>>><br>
>>>> coredump_dir /var/spool/squid<br>
>>>><br>
>>>> refresh_pattern ^ftp: 144020% 10080<br>
>>>><br>
>>>> refresh_pattern ^gopher:14400%1440<br>
>>>><br>
>>>> refresh_pattern -i (/cgi-bin/|\?) 0 0%0<br>
>>>><br>
>>>> refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0<br>
>>>> refresh-ims<br>
>>>><br>
>>>> refresh_pattern \/Release(|\.gpg)$ 0 0% 0 refresh-ims<br>
>>>><br>
>>>> refresh_pattern \/InRelease$ 0 0% 0 refresh-ims<br>
>>>><br>
>>>> refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 <br>
>>>> refresh-ims<br>
>>>><br>
>>>> refresh_pattern . 0 20% 4320<br>
>>>><br>
>>>><br>
>>>> shutdown_lifetime 1 seconds<br>
>>>><br>
>>>> logfile_rotate 0<br>
>>>><br>
>>>> max_filedescriptors 16384<br>
>>>><br>
>>>> dns_nameservers 8.8.8.8 8.8.4.4 1.1.1.1<br>
>>>><br>
>>>> cache deny all<br>
>>>><br>
>>>> cache_dir null /tmp<br>
>>>><br>
>>>> via off<br>
>>>><br>
>>>> forwarded_for off<br>
>>>><br>
>>>> request_header_access From deny all<br>
>>>><br>
>>>> request_header_access Server deny all<br>
>>>><br>
>>>> request_header_access WWW-Authenticate deny all<br>
>>>><br>
>>>> request_header_access Link deny all<br>
>>>><br>
>>>> request_header_access Cache-Control deny all<br>
>>>><br>
>>>> request_header_access Proxy-Connection deny all<br>
>>>><br>
>>>> request_header_access X-Cache deny all<br>
>>>><br>
>>>> request_header_access X-Cache-Lookup deny all<br>
>>>><br>
>>>> request_header_access Via deny all<br>
>>>><br>
>>>> request_header_access X-Forwarded-For deny all<br>
>>>><br>
>>>> request_header_access Pragma deny all<br>
>>>><br>
>>>> request_header_access Keep-Alive deny all<br>
>>>><br>
>>>> dns_v4_first on<br>
>>>><br>
>>>><br>
>>>> Example service file:<br>
>>>><br>
>>>> ## Copyright (C) 1996-2020 The Squid Software Foundation and<br>
>>>> contributors<br>
>>>><br>
>>>> ##<br>
>>>><br>
>>>> ## Squid software is distributed under GPLv2+ license and includes<br>
>>>><br>
>>>> ## contributions from numerous individuals and organizations.<br>
>>>><br>
>>>> ## Please see the COPYING and CONTRIBUTORS files for details.<br>
>>>><br>
>>>> ##<br>
>>>><br>
>>>><br>
>>>> [Unit]<br>
>>>><br>
>>>> Description=Squid Web Proxy Server<br>
>>>><br>
>>>> Documentation=man:squid(8)<br>
>>>><br>
>>>> After=network.target network-online.target nss-lookup.target<br>
>>>><br>
>>>><br>
>>>> [Service]<br>
>>>><br>
>>>> Type=forking<br>
>>>><br>
>>>> PIDFile=/var/run/squid2.pid<br>
>>>><br>
>>>> ExecStartPre=/usr/sbin/squid --foreground -z -f /etc/squid/squid2.conf<br>
>>>><br>
>>>> ExecStart=/usr/sbin/squid -sYC -f /etc/squid/squid2.conf<br>
>>>><br>
>>>> ExecReload=/bin/kill -HUP $MAINPID<br>
>>>><br>
>>>> KillMode=mixed<br>
>>>><br>
>>>><br>
>>>> [Install]<br>
>>>><br>
>>>> WantedBy=multi-user.target<br>
>>>><br>
>>>><br>
>>>><br>
>>>> Permissions:<br>
>>>><br>
>>>> ➜ ls -alt /etc/squid/<br>
>>>> total 128<br>
>>>> drwxr-xr-x 2 root root 4096 Sep 6 11:33 .<br>
>>>> -rw-r--r-- 1 root root 2831 Sep 6 11:33 squid7.conf<br>
>>>> drwxr-xr-x 116 root root 4096 Sep 6 11:33 ..<br>
>>>> -rw-r--r-- 1 root root 2830 Sep 6 11:33 squid2.conf<br>
>>>> -rw-r--r-- 1 root root 2836 Sep 6 11:33 squid13.conf<br>
>>>> -rw-r--r-- 1 root root 2836 Sep 6 11:32 squid23.conf<br>
>>>> -rw-r--r-- 1 root root 2836 Sep 6 11:32 squid19.conf<br>
>>>> -rw-r--r-- 1 root root 2832 Sep 6 11:32 squid1.conf<br>
>>>> -rw-r--r-- 1 root root 2836 Sep 6 11:32 squid17.conf<br>
>>>> -rw-r--r-- 1 root root 2832 Sep 6 11:31 squid4.conf<br>
>>>> -rw-r--r-- 1 root root 2834 Sep 6 11:31 squid21.conf<br>
>>>> -rw-r--r-- 1 root root 2833 Sep 6 11:31 squid25.conf<br>
>>>> -rw-r--r-- 1 root root 2834 Sep 6 11:31 squid12.conf<br>
>>>> -rw-r--r-- 1 root root 2832 Sep 6 11:31 squid3.conf<br>
>>>> -rw-r--r-- 1 root root 2836 Sep 6 11:30 squid10.conf<br>
>>>> -rw-r--r-- 1 root root 2835 Sep 6 11:30 squid11.conf<br>
>>>> -rw-r--r-- 1 root root 2833 Sep 6 11:30 squid18.conf<br>
>>>> -rw-r--r-- 1 root root 2830 Sep 6 11:30 squid8.conf<br>
>>>> -rw-r--r-- 1 root root 2830 Sep 6 11:30 squid6.conf<br>
>>>> -rw-r--r-- 1 root root 2833 Sep 6 11:30 squid28.conf<br>
>>>> -rw-r--r-- 1 root root 2830 Sep 6 11:25 squid9.conf<br>
>>>> -rw-r--r-- 1 root root 2836 Sep 6 11:25 squid24.conf<br>
>>>> -rw-r--r-- 1 root root 2835 Sep 6 11:25 squid22.conf<br>
>>>> -rw-r--r-- 1 root root 2837 Sep 6 11:25 squid20.conf<br>
>>>> -rw-r--r-- 1 root root 2836 Sep 6 11:25 squid16.conf<br>
>>>> -rw-r--r-- 1 root root 2835 Sep 6 11:25 squid15.conf<br>
>>>> -rw-r--r-- 1 root root 2836 Sep 6 11:25 squid14.conf<br>
>>>> -rw-r--r-- 1 root root 2831 Sep 6 11:25 squid5.conf<br>
>>>> -rw-r--r-- 1 root root 2833 Sep 6 11:25 squid27.conf<br>
>>>> -rw-r--r-- 1 root root 2835 Sep 6 11:25 squid26.conf<br>
>>>> -rw-r--r-- 1 root root 2835 Sep 6 11:25 squid30.conf<br>
>>>> -rw-r--r-- 1 root root 2835 Sep 6 11:25 squid29.conf<br>
>>>><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> squid-users mailing list<br>
>>>> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank" rel="noreferrer">squid-users@lists.squid-cache.org</a><br>
>>>> <a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
>>><br>
>>> _______________________________________________<br>
>>> squid-users mailing list<br>
>>> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank" rel="noreferrer">squid-users@lists.squid-cache.org</a><br>
>>> <a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
> <br>
> _______________________________________________<br>
> squid-users mailing list<br>
> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank" rel="noreferrer">squid-users@lists.squid-cache.org</a><br>
> <a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
<br>
_______________________________________________<br>
squid-users mailing list<br>
<a href="mailto:squid-users@lists.squid-cache.org" target="_blank" rel="noreferrer">squid-users@lists.squid-cache.org</a><br>
<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
</blockquote></div></div></div>