[squid-users] Volume quota management availablity

[Amos Jeffries]

On 19/03/18 23:03, Anoop Sreedharan wrote:
> Dear Team,
> We have an IT environment catering to educational institute wherein we
> have approx more than 1000 users accessing the internet.
> 
> having a volume based internet subscription, we are in need to have a
> solution wherein i need to restrict users to a certain volume of quota
> per month and upon crossing that threshold, need to either stop their
> access or throttle their bandwidth speed.
> 
> following is the scenario. having an internet link of 50Mbps in my campus
> 
> 1. users have to be authenticated via Active Directory  -- i.e. users in
> a certain AD group should only get access to internet
> 2. should be able to define a volume threshold ( e.g 100GB per group/per
> user)
> 3. upon exhaustion of the volume the user bandwidth should b throttled
> to, say, 256Kbps. OR block internet access to that user completely.
> 4. this volume calculation should be done for both HTTP and HTTPS based
> session.
> 5. should be able to generate a monthly report showcasing the volume
> consumed by specific user during a specific time-frame by showing the
> spread of the volume distributed within websites visited/downloaded from.
>  
> Kindly help to suggest this could be possible with Squid. 
> I am open to using some log analytics mechanism like sarg or anything
> similar for reporting.

Quota is not a concept easily applied to HTTP messaging since it is a
stateless protocol and operates in terms of entire messages - not
packets or bytes. As such there is intentionally no mechanism to
maintain statefulness between transactions for quota controls to use in
Squid.

There is also a rather lot of traffic details outside of HTTP an unknown
to Squid which greatly affect the relationship between what Squid sees
as bandwidth and what actually occurs "on wire". That all makes the OS
networking stack a much better place to do such management.

Most OS these days provide very capable tools for QoS bandwidth
management. Squid provides configuration features to integrate with
those, delivering packet TOS markings per-transaction or per-message for
the machines OS systems to utilize in their QoS flow identification and
accounting.



Log analysis (eg SARG) and helpers are other possibilities that worked
in the past ...

 BUT these methods have always suffered from the problem of only
accounting for traffic usage at the end of a completed HTTP transaction
and authorizing users only at the beginning. The difference can see
large amounts of over-usage and CONNECT tunnels are the worst-case
scenario there as they may last for days/weeks with "infinite" amount of
traffic usage meanwhile.

 Added to those problems we now face most traffic being HTTPS ... which
goes through proxies via CONNECT tunnels. So much for those ways of
doing quotas.


Amos