[squid-users] Fwd: cache_peer_access by dynamic ACL
Alexeyяр Gruzdov
my.shellac at gmail.com
Thu Apr 20 17:14:36 UTC 2023
Ok. The last issues fixed. Thank you !
Tell me please If I right understood I could to get answer like
"name=value" from my ACL ext script, instead of "OK" or "ERR", right?
And does it means - I could to get answer depends from what users
authorises in to proxy.
For example:
If user "Jon" - the my ACL will check the policy in the DB and send
answer like "proxy=g1" to squid, or if user is "Jack" - answer will be like
"proxy=all"
and I will have ACL for this
acl proxy_g1_marked.acl note proxy g1
acl proxy_all_marked.acl note proxy all
will that correct ?
чт, 20 апр. 2023 г. в 19:50, Alex Rousskov <rousskov at measurement-factory.com
>:
> On 4/20/23 10:54, Alexeyяр Gruzdov wrote:
> > The cpu is around 100% even no any requests is going....
>
> Then the problem is most likely in your script. Foe example, the script
> actively doing something while there are no Squid requests to work on
> (instead of blocking while waiting for the next Squid request).
>
>
> > For now I left just one cache_peer in configuration and got the new
> error:
> >
> > commBind Cannot bind socket FD 28 to [::]: (13) Permission denied
>
> Could be one of the misconfigurations discussed at
>
> https://wiki.squid-cache.org/Features/SmpScale#cannot-bind-socket-fd-nn-to--13-permission-denied
>
>
> HTH,
>
> Alex.
>
>
> > чт, 20 апр. 2023 г. в 17:21, Alex Rousskov:
> >
> > On 4/20/23 04:23, Alexeyяр Gruzdov wrote:
> >
> > > cache_peer peerG1.com parent 40001 0 no-query no-digest
> name=peerG1
> > >
> > > external_acl_type ext_proxy_g1_type %LOGIN %DST
> /usr/local/bin/g1.py
> > >
> > > acl proxy_g1_ext_mark_acl ext_proxy_g1_type
> > >
> > > acl proxy_g1_ext_marked_acl annotate_transaction proxy=g1
> > >
> > > acl proxy_peerG1_acl note proxy g1
> > >
> > > http_access deny
> proxy_g1_ext_mark_acl proxy_g1_ext_marked_acl !all
> > > .....
> > > others http_access rules
> > >
> > > And this above works.
> >
> > Glad to hear that. ( If others are going to use the above as a
> guiding
> > example, I would recommend naming these ACLs very differently, but
> that
> > is not important to Squid. )
> >
> >
> > > BUT
> > > I am worried about why this my external script for ACL type
> > loads the
> > > one of core of CPU to 100%.....???
> >
> > External ACL caching aside, the script will be contacted once for
> every
> > Squid transaction. Does your script CPU usage go down to zero when
> > there
> > is no traffic? If not, then there is a bug in the script itself.
> >
> > If you use the script from the command line, without Squid, does it
> > consume a lot of CPU and/or take a lot of time per fake query? You
> can
> > adjust the script to log the real query (when the script is used by
> > Squid), so that you can easily replicate that query when running the
> > script without Squid...
> >
> > The cache key in your case is (the expansion of) "%LOGIN %DST". It is
> > enabled by default IIRC. Look for "cache" related options at
> > http://www.squid-cache.org/Doc/config/external_acl_type/
> > <http://www.squid-cache.org/Doc/config/external_acl_type/>
> >
> >
> > > ( I used three of workers in config,
> > > but I can see a six process called like my external helper
> > script, looks
> > > like squid runs x2 process for external ACL )
> >
> > See external_acl_type children-* options:
> > http://www.squid-cache.org/Doc/config/external_acl_type/
> > <http://www.squid-cache.org/Doc/config/external_acl_type/>
> >
> > In most environments, I recommend setting all three of them to the
> same
> > value. Please note that these options are not SMP-aware (yet), so
> Squid
> > will _not_ divide their values by the number of workers and give each
> > worker as many children as you state in squid.conf.
> >
> >
> > > Because if I will put the one more group of users (that must to
> use
> > > another cache_peer ) - I will need to create one more external
> > script
> > > that will making to check an existed users from an other DB table
> >
> > Once you get the basic setup above working for one group to your
> > satisfaction, I would recommend migrating from (one script and one
> > matching annotate_transaction ACL) per group to a single script for
> all
> > groups. That single external ACL script will send the right
> > annotation(s) to Squid.
> >
> >
> > HTH,
> >
> > Alex.
> >
> >
> > > ср, 19 апр. 2023 г. в 22:39, Alex Rousskov:
> > >
> > > On 4/19/23 13:30, Alexeyяр Gruzdov wrote:
> > >
> > > > cache_peer peerG1.com parent 40001 0 no-query no-digest
> > name=peerG1
> > >
> > > > external_acl_type ext_proxy_g1_type %LOGIN %DST
> > /usr/local/bin/g1.py
> > >
> > > > acl proxy_g1_ext_acl ext_proxy_g1_type
> > >
> > > OK. I assume that /usr/local/bin/g1.py will only match users
> that
> > > should
> > > go to cache_peer called peerG1.
> > >
> > >
> > > > acl proxy_g1_ext_acl_mark annotate_transaction proxy=g1
> > >
> > > Please note that the name of this annotate_transaction ACL --
> > > "proxy_g1_ext_acl_mark" -- implies a relationship to the
> > external ACL
> > > named "proxy_g1_ext_acl", but there is no such relationship.
> > Squid does
> > > not care about ACL names, but this naming problem may
> indicate a
> > > misunderstanding. To follow your naming scheme, this ACL
> > should be
> > > called something like "proxy_g1_mark_acl" or
> > "mark_for_proxy_g1_acl".
> > >
> > >
> > > > acl proxy_peerG1_acl note proxy g1
> > >
> > > OK. FWIW, a more consistent ACL name would have been
> > > "proxy_g1_marked_acl" or "marked_for_proxy_g1_acl". Again,
> > Squid does
> > > not really care about these names, so use whatever you think
> is
> > > consistent/meaningful/etc.
> > >
> > >
> > > > http_access deny proxy_g1_ext_acl !all
> > >
> > > This line has no (positive) effect. Squid will evaluate the
> > external
> > > ACL, but since the rule, as a whole, will never match due to
> > "!all",
> > > and
> > > since the external ACL has no (relevant) side effects, you
> > can just
> > > delete this line from your configuration.
> > >
> > > Needless to say, if you delete this line, then
> > proxy_g1_ext_acl will be
> > > unused, which should tell you that this configuration is not
> > doing what
> > > you want. See below for a fix recommendation.
> > >
> > >
> > > > http_access deny proxy_g1_ext_acl_mark !all
> > >
> > > This line will mark _all_ transactions. You only want to mark
> > > transactions that also matched proxy_g1_ext_acl. That "b only
> > if a"
> > > logic is accomplished by using _both_ ACLs in the same rule:
> > >
> > > http_access deny proxy_g1_ext_acl proxy_g1_ext_acl_mark
> !all
> > >
> > > With the above http_access rule (instead of the earlier two),
> > Squid
> > > will
> > > evaluate the external ACL, and, if it matches, Squid will
> > also evaluate
> > > the annotation-setting ACL. The whole rule will then be
> > rejected due to
> > > "!all", but not until it annotates the transaction (if the
> > external ACL
> > > matches). Again, in this sketch, we are using this rule for
> its
> > > annotation side effect only.
> > >
> > >
> > > > And this works like I need now....
> > >
> > > AFAICT, if the tests indicate that this configuration works,
> > then the
> > > tests are broken. IMHO, you should fix the tests (while you
> > have a
> > > broken configuration that can be used to test the tests)
> before
> > > proceeding with the configuration fix.
> > >
> > >
> > > HTH,
> > >
> > > Alex.
> > > P.S. Please keep this email thread on squid-users instead of
> > responding
> > > to me personally.
> > >
> > >
> > >
> > >
> > > > ср, 19 апр. 2023 г. в 21:01, Alexeyяр Gruzdov:
> > > >
> > > > so, ok - Lets limit just to one cache peer and one
> single
> > > ACL (just
> > > > to understand the logic):
> > > >
> > > > cache_peer peerG1.com parent 40001 0 no-query
> no-digest
> > > name=peerG1
> > > >
> > > > external_acl_type ext_proxy_g1_type %LOGIN %DST
> > > > /usr/local/bin/g1.py (this will answer "OK" or
> "ERR",
> > > depends if
> > > > user consists in DB)
> > > >
> > > > acl proxy_g1_ext_acl
> > ext_proxy_g1_type annotate_transaction
> > > > proxy=g1 (If I right understood here is a key point
> > of how
> > > to add
> > > > the tag to transaction related with user)
> > > > acl proxy_peerG1_acl note proxy g1 (here we create
> > the ACL
> > > based
> > > > on the tag and this is fast ACL yet and we should to
> > use it in
> > > > cache_peer_access)
> > > >
> > > >
> > > > http_access deny proxy_g1_ext_acl !all
> > > > ......<others http access rules>
> > > >
> > > >
> > > > cache_peer_access peerG1 allow proxy_peerG1_acl
> > > > cache_peer_access peerG1 deny all
> > > >
> > > > Is that correct ?
> > > >
> > > > вт, 18 апр. 2023 г. в 23:44, Alex Rousskov
> > > > <rousskov at measurement-factory.com
> > <mailto:rousskov at measurement-factory.com>
> > > <mailto:rousskov at measurement-factory.com
> > <mailto:rousskov at measurement-factory.com>>
> > > > <mailto:rousskov at measurement-factory.com
> > <mailto:rousskov at measurement-factory.com>
> > > <mailto:rousskov at measurement-factory.com
> > <mailto:rousskov at measurement-factory.com>>>>:
> > > >
> > > > On 4/18/23 11:41, Alexeyяр Gruzdov wrote:
> > > >
> > > > > Could you explain me how the annotation
> transaction
> > > works and
> > > > how it
> > > > > related to acl that I could to use with
> cache_peers
> > > >
> > > > Transactions have a (possibly empty) set of
> name=value
> > > annotations.
> > > >
> > > > During Squid configuration time, Squid parses all
> ACL
> > > > declarations in
> > > > your configuration file. When Squid parses an
> > > > annotation_transaction ACL
> > > > declaration, Squid remembers what transaction
> > annotation
> > > to add
> > > > in the
> > > > future, [every time] when that ACL is evaluated
> (e.g.,
> > > used in
> > > > http_access rule that Squid reaches during
> transaction
> > > processing).
> > > >
> > > > When evaluated, an "annotation_transaction" ACL
> simply
> > > adds the
> > > > previously configured annotation to the current
> > > transaction and
> > > > returns
> > > > a "yes, this transaction matches" result.
> > > >
> > > > When evaluated, a "note" ACL returns a "yes, this
> > transaction
> > > > matches"
> > > > result if and only if the current transaction
> > already has the
> > > > matching
> > > > annotation. This ACL does not modify the set of
> > transaction
> > > > annotations.
> > > >
> > > > The combination of annotate_transaction and note
> ACLs
> > > allows you to
> > > > annotate a transaction at one time and check
> > previously set
> > > > transaction
> > > > annotations at another time. The timing and
> > meaning of those
> > > > annotations
> > > > are up to you.
> > > >
> > > >
> > > > > ok! Lets look to my case example:
> > > >
> > > > > cache_peer peerG1.com parent 40001 0 no-query
> > no-digest
> > > > name=peerG1 round-robin
> > > >
> > > > > cache_peer_access peerG1 allow proxy_peerG1_acl
> > > > > cache_peer_access peerG1 allow proxy_all_acl
> > > > > cache_peer_access peerG1 deny all
> > > >
> > > > > acl proxy_peerG1_acl proxy_auth
> > "../users.peerG1.txt"
> > > > > acl proxy_all_acl proxy_auth
> "../users.all.txt"
> > > >
> > > > [ I added the missing "acl " directive to the
> > above ACL
> > > > declarations and
> > > > stripped rules for two out of three cache_peers ]
> > > >
> > > > As you know, the above cache_peer_access
> > configuration is not
> > > > supported
> > > > because it uses "slow" proxy_auth ACLs in
> > cache_peer_access
> > > > directives
> > > > that only support "fast" ACLs. It does not matter
> > (to me),
> > > > whether the
> > > > above appears to "work" in some environments. YMMV.
> > > >
> > > > To fix this problem, we can use http_access rules
> to
> > > essentially
> > > > remember proxy_auth evaluation results (at
> http_access
> > > > evaluation time)
> > > > as transaction annotations. Here is an untested
> > sketch that
> > > > omits other
> > > > (important but irrelevant here) http_access rules
> > and assumes
> > > > that these
> > > > sketched http_access rules _are_ evaluated:
> > > >
> > > > # if proxy_peerG1_acl matches, evaluate
> > mark_for_peerG1
> > > > http_access deny proxy_peerG1_acl
> > mark_for_peerG1 !all
> > > >
> > > > # if proxy_all_acl matches, evaluate
> > mark_for_all_peers
> > > > http_access deny proxy_all_acl
> > mark_for_all_peers !all
> > > >
> > > >
> > > > Now we can use those remembered proxy_... acl
> > evaluation
> > > results
> > > > (i.e.
> > > > we can check for the matching annotations) in
> > > cache_peer_access
> > > > rules:
> > > >
> > > > cache_peer_access peerG1 allow
> marked_for_peerG1
> > > > cache_peer_access peerG1 allow
> > marked_for_all_peers
> > > > cache_peer_access peerG1 deny all
> > > >
> > > >
> > > > where the new ACLs mentioned above are declared
> along
> > > these lines:
> > > >
> > > > acl mark_for_peerG1 annotate_transaction
> > for_peer_=G1
> > > > acl mark_for_all_peers annotate_transaction
> > > for_all_peers_=true
> > > >
> > > > acl marked_for_peerG1 note for_peer_ G1
> > > > acl marked_for_all_peers note for_all_peers_
> true
> > > >
> > > > This can probably be simplified further by using
> > > for_peer_=ALL
> > > > instead
> > > > of for_all_peers_=true annotation, but I wanted to
> > > preserve the
> > > > symmetry
> > > > with your original configuration.
> > > >
> > > >
> > > > > And these all works like I need, But - once I am
> > > changing a
> > > > list of
> > > > > users (add or remove) - I need to use "squid -k
> > > > reconfigure"...... but
> > > > > of course better to go without this reconfigure
> > > >
> > > > One can avoid reconfiguration using an external ACL
> > > script that
> > > > gives
> > > > Squid the right for_peer_=... annotations (instead
> > of using
> > > > "constant"
> > > > or "hard-coded" annotate_transaction ACLs to store
> > the same
> > > > annotations).
> > > >
> > > > However, it may be better to make the above sketch
> > to work
> > > > _before_ you
> > > > replace mark_for_peerG1 ACLs/rules with an external
> > > > mark_for_the_right_peer ACL.
> > > >
> > > >
> > > > HTH,
> > > >
> > > > Alex.
> > > > P.S. This thread continues the discussion started
> at
> > > > https://bugs.squid-cache.org/show_bug.cgi?id=5268
> > <https://bugs.squid-cache.org/show_bug.cgi?id=5268>
> > > <https://bugs.squid-cache.org/show_bug.cgi?id=5268
> > <https://bugs.squid-cache.org/show_bug.cgi?id=5268>>
> > > > <https://bugs.squid-cache.org/show_bug.cgi?id=5268
> > <https://bugs.squid-cache.org/show_bug.cgi?id=5268>
> > > <https://bugs.squid-cache.org/show_bug.cgi?id=5268
> > <https://bugs.squid-cache.org/show_bug.cgi?id=5268>>>
> > > >
> > > > _______________________________________________
> > > > squid-users mailing list
> > > > squid-users at lists.squid-cache.org
> > <mailto:squid-users at lists.squid-cache.org>
> > > <mailto:squid-users at lists.squid-cache.org
> > <mailto:squid-users at lists.squid-cache.org>>
> > > > <mailto:squid-users at lists.squid-cache.org
> > <mailto:squid-users at lists.squid-cache.org>
> > > <mailto:squid-users at lists.squid-cache.org
> > <mailto:squid-users at lists.squid-cache.org>>>
> > > > http://lists.squid-cache.org/listinfo/squid-users
> > <http://lists.squid-cache.org/listinfo/squid-users>
> > > <http://lists.squid-cache.org/listinfo/squid-users
> > <http://lists.squid-cache.org/listinfo/squid-users>>
> > > > <http://lists.squid-cache.org/listinfo/squid-users
> > <http://lists.squid-cache.org/listinfo/squid-users>
> > > <http://lists.squid-cache.org/listinfo/squid-users
> > <http://lists.squid-cache.org/listinfo/squid-users>>>
> > > >
> > > >
> > > >
> > > > --
> > > > С уважением к Вам
> > > > Алексей
> > > > +79043828661
> > > > 620000 г.Екатеринбург 2022
> > > >
> > > >
> > > >
> > > > --
> > > > С уважением к Вам
> > > > Алексей
> > > > +79043828661
> > > > 620000 г.Екатеринбург 2022
> > > >
> > >
> > >
> > >
> > > --
> > > С уважением к Вам
> > > Алексей
> > > +79043828661
> > > 620000 г.Екатеринбург 2022
> > >
> > >
> > > _______________________________________________
> > > squid-users mailing list
> > > squid-users at lists.squid-cache.org
> > <mailto:squid-users at lists.squid-cache.org>
> > > http://lists.squid-cache.org/listinfo/squid-users
> > <http://lists.squid-cache.org/listinfo/squid-users>
> >
> > _______________________________________________
> > squid-users mailing list
> > squid-users at lists.squid-cache.org
> > <mailto:squid-users at lists.squid-cache.org>
> > http://lists.squid-cache.org/listinfo/squid-users
> > <http://lists.squid-cache.org/listinfo/squid-users>
> >
> >
> > _______________________________________________
> > squid-users mailing list
> > squid-users at lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20230420/064c81fc/attachment-0001.htm>
More information about the squid-users
mailing list