[squid-users] auth, ssl bump and analyzing logs

[Amos Jeffries]

On 14/07/2016 8:38 p.m., Marko Cupać wrote:
> On Thu, 14 Jul 2016 20:23:03 +1200
> Amos Jeffries <squid3 at treenet.co.nz> wrote:
> 
>> On 14/07/2016 7:45 p.m., Marko Cupać wrote:
>>> Hi,
>>>
>>> is there a way to parse squid access.log in a way that only traffic
>>> that was served to clients count?
>>
>> Unless you are using a custom log. That is the only data that gets
>> logged in the size field of access.log.
> 
> I am using default squid format.
> 
>> There are several more types of transaction status than HIT, MISS,
>> ERROR in modern Squid logs.
> 
> I can also get those expanded:
> 
> # Incoming TCP-requests by status
> status                             request      %    Byte       %   sec  kB/sec 
> --------------------------------- --------- ------ -------- ------ ---- ------- 
> HIT                                   97006   6.28 1917953K   4.27    0  168.55 
>  TCP_MEM_HIT                          36766   2.38  776934K   1.73    0  237.31 
>  TCP_HIT                              36640   2.37 1086401K   2.42    0  161.41 
>  TCP_IMS_HIT                          23389   1.51 12253316   0.03    0   80.17 
>  TCP_HIT_ABORTED                        163   0.01 39804662   0.09    7   34.80 
>  TCP_MEM_HIT_ABORTED                     48   0.00  3871446   0.01    2   34.85 
> MISS                                 750043  48.54   18636M  42.46    2   15.74 
>  TCP_MISS                            567543  36.73   15432M  35.16    1   20.55 
>  TAG_NONE                            151551   9.81   613969   0.00    2    0.00 

TAG_NONE is not one of these three categories.


>  TCP_REFRESH_UNMODIFIED               16206   1.05  147799K   0.33    0   99.75 

These ^^ are HITs.

>  TCP_MISS_ABORTED                      5847   0.38 2835049K   6.31   15   31.59 
>  TCP_CLIENT_REFRESH_MISS               5433   0.35  101448K   0.23    0   85.84 
>  TCP_TUNNEL                            1899   0.12  151616K   0.34   24    3.27 

TUNNEL is not one of these three categories.

>  TCP_REFRESH_MODIFIED                  1327   0.09 38878113   0.08    0   79.35 

These ^^ are both a HIT and a MISS. :-P

>  TCP_SWAPFAIL_MISS                      185   0.01  1702107   0.00    0   65.96 
>  TCP_CLIENT_REFRESH_MISS_ABORTED         28   0.00   542744   0.00    1   23.00 
>  TCP_MISS_TIMEDOUT                        9   0.00  2074692   0.00 1182    0.19 
>  TCP_REFRESH_UNMODIFIED_ABORTED           8   0.00  2186179   0.00   12   22.89 

These ^^ are HITs as well.

>  TCP_REFRESH_MODIFIED_ABORTED             4   0.00   158836   0.00    1   30.43 

see REFRESH_MODIFIED.

>  TCP_SWAPFAIL_MISS_ABORTED                3   0.00        0   0.00    0    0.00 
> ERROR                                698141  45.18   23387M  53.28    1   48.90 
>  TCP_DENIED                          553410  35.82   13353M  30.42    0 1247.84 
>  TAG_NONE                             82143   5.32 7500058K  16.69    1   67.05 

see earlier TAG_NONE.

>  TCP_DENIED_ABORTED                   54691   3.54 2623701K   5.84    0  822.03 
>  TCP_MISS                              6011   0.39  149387K   0.33    0  458.36 

TCP_MISS is an error ?

>  TAG_NONE_ABORTED                      1783   0.12        0   0.00   35    0.00 
>  TCP_MISS_ABORTED                        54   0.00  1227715   0.00    1   36.09 
>  TAG_NONE_TIMEDOUT                       38   0.00        0   0.00 7800    0.00 
>  TCP_DENIED_TIMEDOUT                     11   0.00   595022   0.00  445    0.12 
> --------------------------------- --------- ------ -------- ------ ---- ------- 
> Sum                                 1545190 100.00   43897M 100.00    1   26.23 
> 
> 
>> Perhapse the problem is that your Calamaris does not handle those
>> other transaction types.
> 
> Do you speak of those in table above, or some additional ones? I am

Yes, those above. Though the way some entries are repeated under MISS
and ERROR implies the status code is also playing a part.

> using calamaris-2.59_2. When you say 'your Calamaris', should I
> conclude there is another Calamaris which can?

Maybe. I don't know much about Calamaris.

Amos