[squid-users] auth, ssl bump and analyzing logs

[Amos Jeffries]

On 14/07/2016 11:59 p.m., Marko Cupać wrote:
> On Thu, 14 Jul 2016 23:27:04 +1200
> Amos Jeffries wrote:
> 
>> On 14/07/2016 8:38 p.m., Marko Cupać wrote:
>>> On Thu, 14 Jul 2016 20:23:03 +1200
>>> Amos Jeffries wrote:
>>>   
>>>> On 14/07/2016 7:45 p.m., Marko Cupać wrote:  
>>>>> Hi,
>>>>>
>>>>> is there a way to parse squid access.log in a way that only
>>>>> traffic that was served to clients count?  
> 
> Amos,
> 
> I'm grateful for the explanation of transaction status types. This info
> is valuable to me as it helps me better understand squid.
> 
> However, this does not solve my problem which would be getting insight
> into accurate per-user bandwidth usage and number of successful requests
> (I mean those which actually resulted in content served to client).
> 

That depends on what you mean by "bandwidth usage" and "content served
to the client".

The various bits of bandwidth between client and Squid is covered quite
well by logformat codes. So you can make a custom log with whatever
values you want to record about that side of each transaction. But the
Squid-to-server side of things is much more sparse.

If you really just need to know how many payload bytes were delivered to
each client by Squid, then adding up all their transactions (HIT, MISS,
ERROR, or otherwise) in your current default log will give you that
total without much trouble.

To get the more accurate total of client usage in both request and reply
directions. You need a custom log with the %st code. That includes the
message headers and the 'upload' payloads sent by the client.


To get Squid-to-server traffic count, there is the %<bs code. But that
one omits all the message and framing overheads of HTTP. Adding other
server bandwidth codes is on my todo list, but way down.


> I don't insist on Calamaris, this is just something I used before and
> am familiar with. Perhaps someone can recommend me some other squid log
> parser which can give me info I need?

I use the SQL DB logging daemon recording %st and some custom PHP web
pages to display reports from the resulting database.

Amos