[squid-users] Problem with access.log and when using SMP

[Giorgi Tepnadze]

 From squid config:

access_log daemon:/var/log/squid3/access.log squid
logfile_daemon /usr/lib/squid3/log_file_daemon

So I am using daemon. At this moment it would take some time to develop 
new daemon, but I have other question, is it possible to give each 
running squid kid different access.log file to write?

George Tepnadze

On 11/30/2014 10:20 AM, Amos Jeffries wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 27/11/2014 1:25 a.m., Giorgi Tepnadze wrote:
>> Hello,
>>
>> I have own squid log parsing software which worked flawlessly
>> before we moved to squid 3.3.8 (old was 2.7). Log parser stopped
>> working. This is the problem I found.
>>
>> Problem lines: 1416971036.802 184393 10.xx.xx.xx TCP_MISS/200 10963
>> CONNECT pixel.facebook.com:1416971036.390      0 10.yy.yy.yy
>> TCP_DENIED/407 3751 CONNECT clients4.google.com:443 - HIER_NONE/-
>> text/html 1416971036.645   7164 10.zz.zz.zz TCP_MISS/200 55438 GET
>> http://s40.radikal.ru/i089/1107/9b/2fbf1779ead9.jpg
>> user at domain.com HIER_DIRECT/81.176.238.141 image/jpeg 443
>> user2 at domainc.com HIER_DIRECT/31.13.93.3 -
>>
>> As it should be: 1416971036.390      0 10.yy.yy.yy TCP_DENIED/407
>> 3751 CONNECT clients4.google.com:443 - HIER_NONE/- text/html
>> 1416971036.802 184393 10.xx.xx.xx TCP_MISS/200 10963 CONNECT
>> pixel.facebook.com:443 user2 at domainc.com HIER_DIRECT/31.13.93.3 -
>> 1416971036.645   7164 10.zz.zz.zz TCP_MISS/200 55438 GET
>> http://s40.radikal.ru/i089/1107/9b/2fbf1779ead9.jpg
>> user at domain.com HIER_DIRECT/81.176.238.141 image/jpeg
>>
>> I think that's because of SMP as I have 4 squid kids running and
>> they all write to same access.log file and there is concurrency
>> problem. Sometimes one is writing log line when previous log line
>> is not finished yet or something like this. Such errors are rare
>> but enough to stop my parser with  error.
>>
>> Is it a bug? Is there any workaround or solution?
> What logging module are you using to write?
>
> Squid log lines are generated into a buffer and emitted in what should
> be close to atomic (but not exactly locking) per-line write() actions.
>
> You could write a custom daemon to receive the log lines and send them
> directly through your parser system as units. This has the added
> benefit of being a lot closer to real-time log processing.
>
> Amos
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (MingW32)
>
> iQEcBAEBAgAGBQJUercvAAoJELJo5wb/XPRjZVEH/RtrMBTH5XfPi6MmIVf2/Yzb
> DZg+ObG64J/SmWMc99YvTL8Bg5SrB4GNKZEt5JM3jKYOkjS1lg7G90JUfwmHi8xI
> xnX/+iayr19QVmdRb5LNZdpgn1spCoodq9RXQKVLKQnxvYVrV5SCnPx51xeevG2g
> X7gBa+l/nltEQy1cc2Dc5UScvMHdXh8tShG8CINX/oyXr6XM0qa8jCe3Tczq966K
> GsEnicp017lwSDVSiXPU866Gf/3xexXGUzPIxlWe3OIbEwp/LP0U63jaF2lUz6cu
> xY10bAUbOTGE76UmhHRSmePlhZcWwsHHwbXy7Z5OQCe2UX/MRIZYzN7GkqSVgf4=
> =iKx3
> -----END PGP SIGNATURE-----
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users