<div dir="ltr">The cpu is around 100% even no any requests is going....<br>For now I left just one cache_peer in configuration and got the new error:<br><br>commBind Cannot bind socket FD 28 to [::]: (13) Permission denied<br><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">чт, 20 апр. 2023 г. в 17:21, Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com">rousskov@measurement-factory.com</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 4/20/23 04:23, Alexeyяр Gruzdov wrote:<br>
<br>
> cache_peer peerG1.com parent 40001 0 no-query no-digest name=peerG1<br>
> <br>
> external_acl_type ext_proxy_g1_type %LOGIN %DST /usr/local/bin/g1.py<br>
> <br>
> acl proxy_g1_ext_mark_acl ext_proxy_g1_type<br>
> <br>
> acl proxy_g1_ext_marked_acl annotate_transaction proxy=g1<br>
> <br>
> acl proxy_peerG1_acl note proxy g1<br>
> <br>
> http_access deny proxy_g1_ext_mark_acl proxy_g1_ext_marked_acl !all<br>
> .....<br>
> others http_access rules<br>
> <br>
> And this above works.<br>
<br>
Glad to hear that. ( If others are going to use the above as a guiding <br>
example, I would recommend naming these ACLs very differently, but that <br>
is not important to Squid. )<br>
<br>
<br>
> BUT<br>
> I am worried about why this my external script for ACL type loads the <br>
> one of core of CPU to 100%.....??? <br>
<br>
External ACL caching aside, the script will be contacted once for every <br>
Squid transaction. Does your script CPU usage go down to zero when there <br>
is no traffic? If not, then there is a bug in the script itself.<br>
<br>
If you use the script from the command line, without Squid, does it <br>
consume a lot of CPU and/or take a lot of time per fake query? You can <br>
adjust the script to log the real query (when the script is used by <br>
Squid), so that you can easily replicate that query when running the <br>
script without Squid...<br>
<br>
The cache key in your case is (the expansion of) "%LOGIN %DST". It is <br>
enabled by default IIRC. Look for "cache" related options at<br>
<a href="http://www.squid-cache.org/Doc/config/external_acl_type/" rel="noreferrer" target="_blank">http://www.squid-cache.org/Doc/config/external_acl_type/</a><br>
<br>
<br>
> ( I used three of workers in config, <br>
> but I can see a six process called like my external helper script, looks <br>
> like squid runs x2 process for external ACL )<br>
<br>
See external_acl_type children-* options:<br>
<a href="http://www.squid-cache.org/Doc/config/external_acl_type/" rel="noreferrer" target="_blank">http://www.squid-cache.org/Doc/config/external_acl_type/</a><br>
<br>
In most environments, I recommend setting all three of them to the same <br>
value. Please note that these options are not SMP-aware (yet), so Squid <br>
will _not_ divide their values by the number of workers and give each <br>
worker as many children as you state in squid.conf.<br>
<br>
<br>
> Because if I will put the one more group of users (that must to use <br>
> another cache_peer ) - I will need to create one more external script <br>
> that will making to check an existed users from an other DB table<br>
<br>
Once you get the basic setup above working for one group to your <br>
satisfaction, I would recommend migrating from (one script and one <br>
matching annotate_transaction ACL) per group to a single script for all <br>
groups. That single external ACL script will send the right <br>
annotation(s) to Squid.<br>
<br>
<br>
HTH,<br>
<br>
Alex.<br>
<br>
<br>
> ср, 19 апр. 2023 г. в 22:39, Alex Rousskov:<br>
> <br>
> On 4/19/23 13:30, Alexeyяр Gruzdov wrote:<br>
> <br>
> > cache_peer peerG1.com parent 40001 0 no-query no-digest name=peerG1<br>
> <br>
> > external_acl_type ext_proxy_g1_type %LOGIN %DST /usr/local/bin/g1.py<br>
> <br>
> > acl proxy_g1_ext_acl ext_proxy_g1_type<br>
> <br>
> OK. I assume that /usr/local/bin/g1.py will only match users that<br>
> should<br>
> go to cache_peer called peerG1.<br>
> <br>
> <br>
> > acl proxy_g1_ext_acl_mark annotate_transaction proxy=g1<br>
> <br>
> Please note that the name of this annotate_transaction ACL --<br>
> "proxy_g1_ext_acl_mark" -- implies a relationship to the external ACL<br>
> named "proxy_g1_ext_acl", but there is no such relationship. Squid does<br>
> not care about ACL names, but this naming problem may indicate a<br>
> misunderstanding. To follow your naming scheme, this ACL should be<br>
> called something like "proxy_g1_mark_acl" or "mark_for_proxy_g1_acl".<br>
> <br>
> <br>
> > acl proxy_peerG1_acl note proxy g1<br>
> <br>
> OK. FWIW, a more consistent ACL name would have been<br>
> "proxy_g1_marked_acl" or "marked_for_proxy_g1_acl". Again, Squid does<br>
> not really care about these names, so use whatever you think is<br>
> consistent/meaningful/etc.<br>
> <br>
> <br>
> > http_access deny proxy_g1_ext_acl !all<br>
> <br>
> This line has no (positive) effect. Squid will evaluate the external<br>
> ACL, but since the rule, as a whole, will never match due to "!all",<br>
> and<br>
> since the external ACL has no (relevant) side effects, you can just<br>
> delete this line from your configuration.<br>
> <br>
> Needless to say, if you delete this line, then proxy_g1_ext_acl will be<br>
> unused, which should tell you that this configuration is not doing what<br>
> you want. See below for a fix recommendation.<br>
> <br>
> <br>
> > http_access deny proxy_g1_ext_acl_mark !all<br>
> <br>
> This line will mark _all_ transactions. You only want to mark<br>
> transactions that also matched proxy_g1_ext_acl. That "b only if a"<br>
> logic is accomplished by using _both_ ACLs in the same rule:<br>
> <br>
> http_access deny proxy_g1_ext_acl proxy_g1_ext_acl_mark !all<br>
> <br>
> With the above http_access rule (instead of the earlier two), Squid<br>
> will<br>
> evaluate the external ACL, and, if it matches, Squid will also evaluate<br>
> the annotation-setting ACL. The whole rule will then be rejected due to<br>
> "!all", but not until it annotates the transaction (if the external ACL<br>
> matches). Again, in this sketch, we are using this rule for its<br>
> annotation side effect only.<br>
> <br>
> <br>
> > And this works like I need now....<br>
> <br>
> AFAICT, if the tests indicate that this configuration works, then the<br>
> tests are broken. IMHO, you should fix the tests (while you have a<br>
> broken configuration that can be used to test the tests) before<br>
> proceeding with the configuration fix.<br>
> <br>
> <br>
> HTH,<br>
> <br>
> Alex.<br>
> P.S. Please keep this email thread on squid-users instead of responding<br>
> to me personally.<br>
> <br>
> <br>
> <br>
> <br>
> > ср, 19 апр. 2023 г. в 21:01, Alexeyяр Gruzdov:<br>
> ><br>
> > so, ok - Lets limit just to one cache peer and one single<br>
> ACL (just<br>
> > to understand the logic):<br>
> ><br>
> > cache_peer peerG1.com parent 40001 0 no-query no-digest<br>
> name=peerG1<br>
> ><br>
> > external_acl_type ext_proxy_g1_type %LOGIN %DST<br>
> > /usr/local/bin/g1.py (this will answer "OK" or "ERR",<br>
> depends if<br>
> > user consists in DB)<br>
> ><br>
> > acl proxy_g1_ext_acl ext_proxy_g1_type annotate_transaction<br>
> > proxy=g1 (If I right understood here is a key point of how<br>
> to add<br>
> > the tag to transaction related with user)<br>
> > acl proxy_peerG1_acl note proxy g1 (here we create the ACL<br>
> based<br>
> > on the tag and this is fast ACL yet and we should to use it in<br>
> > cache_peer_access)<br>
> ><br>
> ><br>
> > http_access deny proxy_g1_ext_acl !all<br>
> > ......<others http access rules><br>
> ><br>
> ><br>
> > cache_peer_access peerG1 allow proxy_peerG1_acl<br>
> > cache_peer_access peerG1 deny all<br>
> ><br>
> > Is that correct ?<br>
> ><br>
> > вт, 18 апр. 2023 г. в 23:44, Alex Rousskov<br>
> > <<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a><br>
> <mailto:<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a>><br>
> > <mailto:<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a><br>
> <mailto:<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a>>>>:<br>
> ><br>
> > On 4/18/23 11:41, Alexeyяр Gruzdov wrote:<br>
> ><br>
> > > Could you explain me how the annotation transaction<br>
> works and<br>
> > how it<br>
> > > related to acl that I could to use with cache_peers<br>
> ><br>
> > Transactions have a (possibly empty) set of name=value<br>
> annotations.<br>
> ><br>
> > During Squid configuration time, Squid parses all ACL<br>
> > declarations in<br>
> > your configuration file. When Squid parses an<br>
> > annotation_transaction ACL<br>
> > declaration, Squid remembers what transaction annotation<br>
> to add<br>
> > in the<br>
> > future, [every time] when that ACL is evaluated (e.g.,<br>
> used in<br>
> > http_access rule that Squid reaches during transaction<br>
> processing).<br>
> ><br>
> > When evaluated, an "annotation_transaction" ACL simply<br>
> adds the<br>
> > previously configured annotation to the current<br>
> transaction and<br>
> > returns<br>
> > a "yes, this transaction matches" result.<br>
> ><br>
> > When evaluated, a "note" ACL returns a "yes, this transaction<br>
> > matches"<br>
> > result if and only if the current transaction already has the<br>
> > matching<br>
> > annotation. This ACL does not modify the set of transaction<br>
> > annotations.<br>
> ><br>
> > The combination of annotate_transaction and note ACLs<br>
> allows you to<br>
> > annotate a transaction at one time and check previously set<br>
> > transaction<br>
> > annotations at another time. The timing and meaning of those<br>
> > annotations<br>
> > are up to you.<br>
> ><br>
> ><br>
> > > ok! Lets look to my case example:<br>
> ><br>
> > > cache_peer peerG1.com parent 40001 0 no-query no-digest<br>
> > name=peerG1 round-robin<br>
> ><br>
> > > cache_peer_access peerG1 allow proxy_peerG1_acl<br>
> > > cache_peer_access peerG1 allow proxy_all_acl<br>
> > > cache_peer_access peerG1 deny all<br>
> ><br>
> > > acl proxy_peerG1_acl proxy_auth "../users.peerG1.txt"<br>
> > > acl proxy_all_acl proxy_auth "../users.all.txt"<br>
> ><br>
> > [ I added the missing "acl " directive to the above ACL<br>
> > declarations and<br>
> > stripped rules for two out of three cache_peers ]<br>
> ><br>
> > As you know, the above cache_peer_access configuration is not<br>
> > supported<br>
> > because it uses "slow" proxy_auth ACLs in cache_peer_access<br>
> > directives<br>
> > that only support "fast" ACLs. It does not matter (to me),<br>
> > whether the<br>
> > above appears to "work" in some environments. YMMV.<br>
> ><br>
> > To fix this problem, we can use http_access rules to<br>
> essentially<br>
> > remember proxy_auth evaluation results (at http_access<br>
> > evaluation time)<br>
> > as transaction annotations. Here is an untested sketch that<br>
> > omits other<br>
> > (important but irrelevant here) http_access rules and assumes<br>
> > that these<br>
> > sketched http_access rules _are_ evaluated:<br>
> ><br>
> > # if proxy_peerG1_acl matches, evaluate mark_for_peerG1<br>
> > http_access deny proxy_peerG1_acl mark_for_peerG1 !all<br>
> ><br>
> > # if proxy_all_acl matches, evaluate mark_for_all_peers<br>
> > http_access deny proxy_all_acl mark_for_all_peers !all<br>
> ><br>
> ><br>
> > Now we can use those remembered proxy_... acl evaluation<br>
> results<br>
> > (i.e.<br>
> > we can check for the matching annotations) in<br>
> cache_peer_access<br>
> > rules:<br>
> ><br>
> > cache_peer_access peerG1 allow marked_for_peerG1<br>
> > cache_peer_access peerG1 allow marked_for_all_peers<br>
> > cache_peer_access peerG1 deny all<br>
> ><br>
> ><br>
> > where the new ACLs mentioned above are declared along<br>
> these lines:<br>
> ><br>
> > acl mark_for_peerG1 annotate_transaction for_peer_=G1<br>
> > acl mark_for_all_peers annotate_transaction<br>
> for_all_peers_=true<br>
> ><br>
> > acl marked_for_peerG1 note for_peer_ G1<br>
> > acl marked_for_all_peers note for_all_peers_ true<br>
> ><br>
> > This can probably be simplified further by using<br>
> for_peer_=ALL<br>
> > instead<br>
> > of for_all_peers_=true annotation, but I wanted to<br>
> preserve the<br>
> > symmetry<br>
> > with your original configuration.<br>
> ><br>
> ><br>
> > > And these all works like I need, But - once I am<br>
> changing a<br>
> > list of<br>
> > > users (add or remove) - I need to use "squid -k<br>
> > reconfigure"...... but<br>
> > > of course better to go without this reconfigure<br>
> ><br>
> > One can avoid reconfiguration using an external ACL<br>
> script that<br>
> > gives<br>
> > Squid the right for_peer_=... annotations (instead of using<br>
> > "constant"<br>
> > or "hard-coded" annotate_transaction ACLs to store the same<br>
> > annotations).<br>
> ><br>
> > However, it may be better to make the above sketch to work<br>
> > _before_ you<br>
> > replace mark_for_peerG1 ACLs/rules with an external<br>
> > mark_for_the_right_peer ACL.<br>
> ><br>
> ><br>
> > HTH,<br>
> ><br>
> > Alex.<br>
> > P.S. This thread continues the discussion started at<br>
> > <a href="https://bugs.squid-cache.org/show_bug.cgi?id=5268" rel="noreferrer" target="_blank">https://bugs.squid-cache.org/show_bug.cgi?id=5268</a><br>
> <<a href="https://bugs.squid-cache.org/show_bug.cgi?id=5268" rel="noreferrer" target="_blank">https://bugs.squid-cache.org/show_bug.cgi?id=5268</a>><br>
> > <<a href="https://bugs.squid-cache.org/show_bug.cgi?id=5268" rel="noreferrer" target="_blank">https://bugs.squid-cache.org/show_bug.cgi?id=5268</a><br>
> <<a href="https://bugs.squid-cache.org/show_bug.cgi?id=5268" rel="noreferrer" target="_blank">https://bugs.squid-cache.org/show_bug.cgi?id=5268</a>>><br>
> ><br>
> > _______________________________________________<br>
> > squid-users mailing list<br>
> > <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
> <mailto:<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a>><br>
> > <mailto:<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
> <mailto:<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a>>><br>
> > <a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
> <<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a>><br>
> > <<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
> <<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a>>><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > С уважением к Вам<br>
> > Алексей<br>
> > +79043828661<br>
> > 620000 г.Екатеринбург 2022<br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > С уважением к Вам<br>
> > Алексей<br>
> > +79043828661<br>
> > 620000 г.Екатеринбург 2022<br>
> ><br>
> <br>
> <br>
> <br>
> -- <br>
> С уважением к Вам<br>
> Алексей<br>
> +79043828661<br>
> 620000 г.Екатеринбург 2022<br>
> <br>
> <br>
> _______________________________________________<br>
> squid-users mailing list<br>
> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
> <a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
<br>
_______________________________________________<br>
squid-users mailing list<br>
<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
</blockquote></div>