[squid-users] Cache Peers and traffic handling

[koshik moshik]

Thank you! Yes, it works fine with 5 peers. So, what would be the best
solution to handle 5000 peers?

On Mon, Apr 12, 2021 at 6:03 PM Alex Rousskov <
rousskov at measurement-factory.com> wrote:

> On 4/10/21 5:03 PM, koshik moshik wrote:
>
> > I am trying to run a Squid proxy Server witth about 5000 cache peers. I
> > am running a dedicated server with 6 cores and 32GB RAM on Ubuntu 16.
> >
> >
> > Could you tell me what else is needed / not needed in my squid.config? I
> > am encountering a high CPU usage and would like to create a very
> > efficient proxy server.
>
> IIRC, Squid code is not optimized for handling a large number of
> cache_peers: Several cache peer selection steps involve linear searches.
>
> I do not know what exactly causes high CPU usage in your environment but
> it could be those linear searches. You can test that (indirectly) by
> decreasing the number of cache_peers from 5000 to, say, 5. That is a
> weak test, of course, because other cache_peer-related overheads could
> be to blame, but I would start there.
>
>
> HTH,
>
> Alex.
>
>
>
> > Down below you can find my squid.config(I deleted the other cache_peer
> > lines):
> >
> > -----------
> >
> > http_port 3128
> >
> > dns_v4_first on
> >
> > acl SSL_ports port 1-65535
> >
> > acl Safe_ports port 1-65535
> >
> > acl CONNECT method CONNECT
> >
> > http_access deny !Safe_ports
> >
> > http_access deny CONNECT !SSL_ports
> >
> > auth_param basic program /usr/lib/squid/basic_ncsa_auth
> /etc/squid/.htpasswd
> >
> > auth_param basic children 5
> >
> > auth_param basic realm Squid Basic Authentication
> >
> > auth_param basic credentialsttl 5 hours
> >
> > acl password proxy_auth REQUIRED
> >
> > http_access allow password
> >
> > #http_access deny all
> >
> > cache allow all
> >
> > never_direct allow all
> >
> > ident_access deny all
> >
> >
> >
> >
> >
> > cache_mem 1 GB
> >
> > maximum_object_size_in_memory 16 MB
> >
> >
> >
> >
> >
> > # Leave coredumps in the first cache dir
> >
> > coredump_dir /var/spool/squid
> >
> >
> > #Rules to anonymize http headers
> >
> > forwarded_for off
> >
> > request_header_access Allow allow all
> >
> > request_header_access Authorization allow all
> >
> > request_header_access WWW-Authenticate allow all
> >
> > request_header_access Proxy-Authorization allow all
> >
> > request_header_access Proxy-Authenticate allow all
> >
> > request_header_access Cache-Control allow all
> >
> > request_header_access Content-Encoding allow all
> >
> > request_header_access Content-Length allow all
> >
> > request_header_access Content-Type allow all
> >
> > request_header_access Date allow all
> >
> > request_header_access Expires allow all
> >
> > request_header_access Host allow all
> >
> > request_header_access If-Modified-Since allow all
> >
> > request_header_access Last-Modified allow all
> >
> > request_header_access Location allow all
> >
> > request_header_access Pragma allow all
> >
> > request_header_access Accept allow all
> >
> > request_header_access Accept-Charset allow all
> >
> > request_header_access Accept-Encoding allow all
> >
> > request_header_access Accept-Language allow all
> >
> > request_header_access Content-Language allow all
> >
> > request_header_access Mime-Version allow all
> >
> > request_header_access Retry-After allow all
> >
> > request_header_access Title allow all
> >
> > request_header_access Connection allow all
> >
> > request_header_access Proxy-Connection allow all
> >
> > request_header_access User-Agent allow all
> >
> > request_header_access Cookie allow all
> >
> > request_header_access All deny all
> >
> >
> >
> >
> >
> > #
> >
> > # Add any of your own refresh_pattern entries above these.
> >
> > #
> >
> > #refresh_pattern ^ftp:           1440    20%     10080
> >
> > #refresh_pattern ^gopher:        1440    0%      1440
> >
> > #refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
> >
> > #refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
> >
> > #refresh_pattern .               0       20%     4320
> >
> >
> > ################################
> >
> > acl me proxy_auth ye-1
> >
> > cache_peer my.proxy.com <http://my.proxy.com/> parent 31280
> > login=user1:password1 no-query name=a1
> >
> > cache_peer_access a1 allow me
> >
> > cache_peer_access a1 deny all
> >
> >
> > _______________________________________________
> > squid-users mailing list
> > squid-users at lists.squid-cache.org
> > http://lists.squid-cache.org/listinfo/squid-users
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210414/79611294/attachment.htm>