[squid-users] Delay Pools or Traffic Shaping per port?!

[Amos Jeffries]

On 5/01/2016 1:41 a.m., Christian Kunkel wrote:
> 
> 
>> Am 04.01.2016 um 12:46 schrieb Amos Jeffries:
>>
>> Squid is limited to 64 listening ports. That can be extended a little in
>> exchange for reducing Squid operating speed, but 200-500 is going very
>> far. This will cause problems with your stated goal of handling Gbps,
>> Squid will need some fine tuning to get near that speed as it is.
> 
> ok. so actually i can run 3 or 4 instances of squid to acomplish my goal of 200 users? lets say the server needs to handle that amount of users and not squid. this would work i guess?
> 

Possibly 2 will be needed to hit 1Gbps. And yes, HTTP is stateless
protocol - as load increases you just add more proxies to handle it. But
that is related to the bps speed, not the number of users. Squid is
limited most by the time it takes to parse and process the HTTP messages.

One Squid can handle many thousands of *users* - when the users are
doing the normal low-ish request rates. Or max out your bandwidth by a
single user doing many thousands of requests.


>>>> Is there any reason you can't use authentication to identify different users?
>>> it does not work with nated ips.
>>
>> Authentication does.
> 
> ok. but i can not use authetication. the main os which will be used to connect to squid can not handle http auth headers. no arp and so on. or lets say it this way: no way to get something unique out of the os to autheticate or authorize on. only thing is used are ports. every user gets a unique port to work with. after login through captive this port is redirected to squid. that ports is actually opened for everyone for 48h. after that time user will see captive to login again. lets say: thats not the best way but the best way i could come up with to do something like authetication. maybe there is a better way but i did not find something to make it better.
>>
>>> it autheticates with ip adress anyway.
>>
>> That is *not* Authentication. That is IP based authorization (access
>> control).
> 
> explained above.

What you explained was *not* authentication, and the reasons given are
not relevant to authentication. Which is a good sign that you do not
understand authentication in HTTP.

That is a clear that you are not going to be able to have it any time
soon with your current level of understanding, whether its possible or
not. So I will continue

>>
>>> so it will limit the ip to 10mbit but behind that ip there are maybe 10 or more ppl.
>>
>> With authentication each of these "ppl" has different credentials and
>> messages using those credentials are used to count the bandwidth shaping
>> towards each user.
>>
>> If your system defines a "user" as being one IP address. Then the IP
>> address is what the traffic needs to be accounted against.
> 
> main problem of NATed users (in my case): their ip adresses changes from time to time or based on their location. so if ip is used for authorization then a big amount gets ahthorized or they need to relogin constantly. not that nice.

With your current setup you are not going to be able to resolve that
problem, or the one about multiple users behind each IP. It is simply
not possible so long as your tie the IP:port details into the definition
of "user" at the captive portal.

Which is why Anthony and I are making such a fuss about checking whether
you can do proper HTTP authentication. Since that has nothing to do with
NAT, IP, port or any of the problematic TCP layer juggling you are doing
in the portal.

>>
>>>>
>>>> What stops users "investigating" the system, and finding out they can get extra 
>>>> bandwidth by using ports which haven't been assigned to them?
>>>
>>> thats the second problem to deal with. there is some kind of a captive portal with login but it opens the port after user autheticates so actually someone else can use that port. so if you have an idea. i would be really thankful :)
>>
>> FYI: the only thing that Squid can do that OS level QoS controls cannot
>> easily do is base its shaping on HTTP message header values (ie the
>> users proxy-auth credentials).
> 
> the os does not really save those credentials. every http request then asks for the credentials. thats to messed up this way.

HTTP is a stateless and multiplexed protocol. Each request is designed
to be a standalone description of how to fetch its reply.

>>
>> Since you want to do this shaping "per-user" without authentication
>> credentials to correctly identify what a single "user" actually is and
>> are instead basing the definition of "user" as stuff coming from an IP
>> address (that IP based authorization) - then OS level QoS controls
>> shaping the traffic based on IP address is the best you are going to
>> get. Despite the NAT related issues.
> 
> see above for ports as unique definition of a user.

Which as you repeatedly have said is not providing you with the unique
portion of the requirement - leaving multiple users behind each IP
and/or ports being re-used by different users.


Now, as to solving your problem:
 I think the best you are going to achieve is to create an external ACL
helper script that receives the client connection IP:port numbers from
Squid, queries the captive portal software and receives a unique tag (or
user= label) assigned for that port, then supplies Squid with the user
name label (almost) as if it had been sent in the HTTP headers.

Amos