[squid-users] squid with random outgoing ip from pool of 1000 ips

[Alex Rousskov]

On 08/27/2016 04:34 PM, --Ahmad-- wrote:

> i guess i need to create probability 1/1000 for each ip.

Yes, but that is _not_ the same as 1/1000 probability for each
tcp_outgoing_address rule, unfortunately. tcp_outgoing_address rules are
evaluated top to bottom until the first matches. If you have N rules and
each rule has a 1/N probability of a match in isolation, then you will
get the following probabilities of a match when the rules are combined:

  rule #0: 1/N   -- good!
  rule #1: (1-1/N) * 1/N  -- which is not 1/N
  rule #2: (1-1/N) * (1-1/N) * 1/N  -- even less 1/N than rule #2 was
  rule #3: (1-1/N) * (1-1/N) * (1-1/N) * 1/N  -- and getting worse!
  ...

To simplify equations, let me denote 1/N as p and (1-1/N) as q. With
your incorrect 1/N ACLs, you get the following probabilities (I am just
rewriting the above using p and q):

  rule #0: p
  rule #1: q * p
  rule #2: q*q * p
  rule #3: q*q*q * p
  ...

If you are still unsure, consider the simple case of just 2 rules
(instead of 1000). You want the second rule to match 50% of the time. If
you give the second rule ACL the same 1/2 probability of a match, then
the second rule will only match 1/4 of the time because it will match
only when the previous rule did _not_ match (1/2) _and_ when its own ACL
matched (1/2): 1/2*1/2 = 1/4.


To compensate for the cumulative effect of rules evaluation, you need
rule i to have p/(q^i) probability of a match (where "q^i" is "q to the
power of i"). With that, you will always get the same probability of a
match (p) for each rule when that rule is evaluated:

  rule #0: p
  rule #1: q * p/q = p
  rule #2: q*q * p/(q*q) = p
  rule #3: q*q*q * p/(q*q*q) = p
  ...
  rule #998: q^998 * p/(q^998) = p

To avoid uncertainty, the last rule (rule #999 in the above notation)
should use the "all" ACL (i.e., it will always match).


> how can i create the randomized acls ???

I suggest writing a script that generates 999 ACLs with correct p/(q^i)
probability and the corresponding tcp_outgoing_address lines to match them.

Please note that computing ~500 random ACL matches for each outgoing
Squid connection (or is it each request?) is not going to be
instantaneous! If you are worried about Squid performance, then you may
want to add custom Squid code to select a random or round-robin IP
address out of a pool of 1000 addresses instead.

[ It is not going to be easy, but if you do it right, the same new
configuration interface and underlying code can then be applied to other
similar tasks in Squid (e.g., selecting one of several load-balanced
ICAP services). In that case, it would be a welcomed feature that may be
officially accepted. If you decide to make this generally useful, then I
recommend getting your configuration design pre-approved on squid-dev
before you implement anything (or before you pay somebody else to
implement it)! ]


> is my settings below is correct ??

No. Your ACL(s) and rule probabilities are wrong. See above.


HTH,

Alex.