<div dir="ltr">First of all thank you for trying to help me. Let me describe my current issue: I have 5000 proxies and would like to hide them. My plan was using another proxy server with 5000 cache peers and 5000 users. Each user would get one peer and one proxy attached to that peer. So basically the outer world would not see my "main proxy" and only the one from the new proxy server. <div><br></div><div>Is there any better solution than cache peers for that? </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 14, 2021 at 8:37 PM Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com">rousskov@measurement-factory.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 4/14/21 2:29 AM, koshik moshik wrote:<br>
> Thank you! Yes, it works fine with 5 peers. So, what would be the best<br>
> solution to handle 5000 peers? <br>
<br>
As you can tell by other responses, you might be asking the wrong<br>
question. However, I will still try to answer your question. The best<br>
option may be to add support for a new Squid configuration parameter<br>
that tells Squid to limit cache_peer candidate accumulation to N peers,<br>
effectively making all those linear searches fast.<br>
<br>
<a href="https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F</a><br>
<br>
Alex.<br>
<br>
<br>
> On Mon, Apr 12, 2021 at 6:03 PM Alex Rousskov wrote:<br>
> <br>
> On 4/10/21 5:03 PM, koshik moshik wrote:<br>
> <br>
> > I am trying to run a Squid proxy Server witth about 5000 cache<br>
> peers. I<br>
> > am running a dedicated server with 6 cores and 32GB RAM on Ubuntu 16. <br>
> ><br>
> ><br>
> > Could you tell me what else is needed / not needed in my<br>
> squid.config? I<br>
> > am encountering a high CPU usage and would like to create a very<br>
> > efficient proxy server.<br>
> <br>
> IIRC, Squid code is not optimized for handling a large number of<br>
> cache_peers: Several cache peer selection steps involve linear searches.<br>
> <br>
> I do not know what exactly causes high CPU usage in your environment but<br>
> it could be those linear searches. You can test that (indirectly) by<br>
> decreasing the number of cache_peers from 5000 to, say, 5. That is a<br>
> weak test, of course, because other cache_peer-related overheads could<br>
> be to blame, but I would start there.<br>
> <br>
> <br>
> HTH,<br>
> <br>
> Alex.<br>
> <br>
> <br>
> <br>
> > Down below you can find my squid.config(I deleted the other cache_peer<br>
> > lines):<br>
> ><br>
> > -----------<br>
> ><br>
> > http_port 3128<br>
> ><br>
> > dns_v4_first on<br>
> ><br>
> > acl SSL_ports port 1-65535<br>
> ><br>
> > acl Safe_ports port 1-65535<br>
> ><br>
> > acl CONNECT method CONNECT<br>
> ><br>
> > http_access deny !Safe_ports<br>
> ><br>
> > http_access deny CONNECT !SSL_ports<br>
> ><br>
> > auth_param basic program /usr/lib/squid/basic_ncsa_auth<br>
> /etc/squid/.htpasswd<br>
> ><br>
> > auth_param basic children 5<br>
> ><br>
> > auth_param basic realm Squid Basic Authentication<br>
> ><br>
> > auth_param basic credentialsttl 5 hours<br>
> ><br>
> > acl password proxy_auth REQUIRED<br>
> ><br>
> > http_access allow password<br>
> ><br>
> > #http_access deny all<br>
> ><br>
> > cache allow all<br>
> ><br>
> > never_direct allow all<br>
> ><br>
> > ident_access deny all<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > cache_mem 1 GB<br>
> ><br>
> > maximum_object_size_in_memory 16 MB<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > # Leave coredumps in the first cache dir<br>
> ><br>
> > coredump_dir /var/spool/squid<br>
> ><br>
> ><br>
> > #Rules to anonymize http headers<br>
> ><br>
> > forwarded_for off<br>
> ><br>
> > request_header_access Allow allow all<br>
> ><br>
> > request_header_access Authorization allow all<br>
> ><br>
> > request_header_access WWW-Authenticate allow all<br>
> ><br>
> > request_header_access Proxy-Authorization allow all<br>
> ><br>
> > request_header_access Proxy-Authenticate allow all<br>
> ><br>
> > request_header_access Cache-Control allow all<br>
> ><br>
> > request_header_access Content-Encoding allow all<br>
> ><br>
> > request_header_access Content-Length allow all<br>
> ><br>
> > request_header_access Content-Type allow all<br>
> ><br>
> > request_header_access Date allow all<br>
> ><br>
> > request_header_access Expires allow all<br>
> ><br>
> > request_header_access Host allow all<br>
> ><br>
> > request_header_access If-Modified-Since allow all<br>
> ><br>
> > request_header_access Last-Modified allow all<br>
> ><br>
> > request_header_access Location allow all<br>
> ><br>
> > request_header_access Pragma allow all<br>
> ><br>
> > request_header_access Accept allow all<br>
> ><br>
> > request_header_access Accept-Charset allow all<br>
> ><br>
> > request_header_access Accept-Encoding allow all<br>
> ><br>
> > request_header_access Accept-Language allow all<br>
> ><br>
> > request_header_access Content-Language allow all<br>
> ><br>
> > request_header_access Mime-Version allow all<br>
> ><br>
> > request_header_access Retry-After allow all<br>
> ><br>
> > request_header_access Title allow all<br>
> ><br>
> > request_header_access Connection allow all<br>
> ><br>
> > request_header_access Proxy-Connection allow all<br>
> ><br>
> > request_header_access User-Agent allow all<br>
> ><br>
> > request_header_access Cookie allow all<br>
> ><br>
> > request_header_access All deny all<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > #<br>
> ><br>
> > # Add any of your own refresh_pattern entries above these.<br>
> ><br>
> > #<br>
> ><br>
> > #refresh_pattern ^ftp: 1440 20% 10080<br>
> ><br>
> > #refresh_pattern ^gopher: 1440 0% 1440<br>
> ><br>
> > #refresh_pattern -i (/cgi-bin/|\?) 0 0% 0<br>
> ><br>
> > #refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880<br>
> ><br>
> > #refresh_pattern . 0 20% 4320<br>
> ><br>
> ><br>
> > ################################<br>
> ><br>
> > acl me proxy_auth ye-1<br>
> ><br>
> > cache_peer <a href="http://my.proxy.com" rel="noreferrer" target="_blank">my.proxy.com</a> <<a href="http://my.proxy.com" rel="noreferrer" target="_blank">http://my.proxy.com</a>><br>
> <<a href="http://my.proxy.com/" rel="noreferrer" target="_blank">http://my.proxy.com/</a> <<a href="http://my.proxy.com/" rel="noreferrer" target="_blank">http://my.proxy.com/</a>>> parent 31280<br>
> > login=user1:password1 no-query name=a1<br>
> ><br>
> > cache_peer_access a1 allow me<br>
> ><br>
> > cache_peer_access a1 deny all<br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > squid-users mailing list<br>
> > <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
> <mailto:<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a>><br>
> > <a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
> <<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a>><br>
> ><br>
> <br>
<br>
</blockquote></div>