[squid-users] Data usage reported in log files

[Yuri Voinov]

11.03.2017 2:57, Antony Stone пишет:
> On Friday 10 March 2017 at 21:50:19, Yuri Voinov wrote:
>
>> Gentlemen, and it never occurred to you that there are other types of
>> traffic besides HTTP / HTTPS, right?
>>
>> DNS, ICMP, other protocols?
> I'm assuming Yosi has been measuring only TCP traffic, but even if he's been 
> measuring everything, I don't think DNS, ICMP and other protocols would add 
> more than 1% on top of HTTP/S, unless (as Marcus suggested) there is also 
> totally-non-Squid traffic on the link being measured.
Come on, sure? Even in L7? Really? Cool story, bro!
>
>
> Antony.
>
>> 11.03.2017 2:44, Yosi Greenfield пишет:
>>> Aha! That could be it. I use sslbump, but not for all users. I'll
>>> check that out, although I think that it's a problem even for bumped
>>> users. Even for bumped users we don't bump all sites, so that really
>>> could be it.
>>>
>>> Thanks!
>>>
>>>
>>> -----Original Message-----
>>> From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On
>>> Behalf Of Marcus Kool
>>> Sent: Friday, March 10, 2017 3:38 PM
>>> To: squid-users at lists.squid-cache.org
>>> Subject: Re: [squid-users] Data usage reported in log files
>>>
>>> On 10/03/17 16:27, Yosi Greenfield wrote:
>>>> Thanks!
>>>>
>>>> Netflow is much larger.
>>>>
>>>> I really want to know exactly what site is costing my users data. Many
>>>> of our users are on metered connections and are paying for overage,
>>>> but I can't tell where that overage is being used. Are they using
>>>> youtube, webmail, wetransfer? I see only a fraction of their actual
>>>> proxy usage in my squid logs.
>>>>
>>>> Data compression would give the opposite result, so that's not what
>>>> I'm seeing.
>>>>
>>>> Any other ideas?
>>> Is there any traffic that is not directed to Squid?
>>>
>>> Do you use ssl-bump in bump mode ?
>>> If not, Squid has no idea how many bytes go through the (HTTPS) tunnels.
>>>
>>> Marcus
>>>
>>>> -----Original Message-----
>>>> From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org]
>>>> On Behalf Of Antony Stone
>>>> Sent: Friday, March 10, 2017 2:21 PM
>>>> To: squid-users at lists.squid-cache.org
>>>> Subject: Re: [squid-users] Data usage reported in log files
>>>>
>>>> On Friday 10 March 2017 at 20:14:36, Yosi Greenfield wrote:
>>>>> Hello all,
>>>>>
>>>>> I'm analyzing my squid logs with sarg, and I see that the number of
>>>>> bytes reported as used by any particular user are often nowhere near
>>>>> the bytes reported by netflow and tcpdump.
>>>> Which is larger?
>>>>
>>>>> I'm trying to trace my users' data usage by site, but I'm unable to
>>>>> do so from the log files because of this.
>>>> Well, what is it you really want to know?
>>>>
>>>> netflow / tcpdump will give you accurate numbers for the quantity of
>>>> data on your Internet link - I assume this is what you're most
>>>> interested in?
>>>> Squid will show you what quantity of data goes to/from the clients,
>>>> but is that really important?
>>>>
>>>>> Can someone please explain to me what I might be missing? Why does
>>>>> squid log report one thing and netflow and tcpdump show something
>>>>> else?
>>>> Data compression?
>>>>
>>>> HTTP responses are often gzipped, so if tcpdump is showing you smaller
>>>> numbers of bytes than Squid reports, that's what I'd look at first.
>>>>
>>>>
>>>> Antony.

-- 
Bugs to the Future
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0x613DEC46.asc
Type: application/pgp-keys
Size: 2437 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20170311/e3e05e85/attachment-0001.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20170311/e3e05e85/attachment-0001.sig>