[squid-users] Cache Peers and traffic handling

[Alex Rousskov]

On 4/10/21 5:03 PM, koshik moshik wrote:

> I am trying to run a Squid proxy Server witth about 5000 cache peers. I
> am running a dedicated server with 6 cores and 32GB RAM on Ubuntu 16. 
> 
> 
> Could you tell me what else is needed / not needed in my squid.config? I
> am encountering a high CPU usage and would like to create a very
> efficient proxy server. 

IIRC, Squid code is not optimized for handling a large number of
cache_peers: Several cache peer selection steps involve linear searches.

I do not know what exactly causes high CPU usage in your environment but
it could be those linear searches. You can test that (indirectly) by
decreasing the number of cache_peers from 5000 to, say, 5. That is a
weak test, of course, because other cache_peer-related overheads could
be to blame, but I would start there.


HTH,

Alex.



> Down below you can find my squid.config(I deleted the other cache_peer
> lines):
> 
> -----------
> 
> http_port 3128
> 
> dns_v4_first on
> 
> acl SSL_ports port 1-65535
> 
> acl Safe_ports port 1-65535
> 
> acl CONNECT method CONNECT
> 
> http_access deny !Safe_ports
> 
> http_access deny CONNECT !SSL_ports
> 
> auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/.htpasswd
> 
> auth_param basic children 5
> 
> auth_param basic realm Squid Basic Authentication
> 
> auth_param basic credentialsttl 5 hours
> 
> acl password proxy_auth REQUIRED
> 
> http_access allow password
> 
> #http_access deny all
> 
> cache allow all
> 
> never_direct allow all
> 
> ident_access deny all
> 
> 
> 
> 
> 
> cache_mem 1 GB
> 
> maximum_object_size_in_memory 16 MB
> 
> 
> 
> 
> 
> # Leave coredumps in the first cache dir
> 
> coredump_dir /var/spool/squid
> 
> 
> #Rules to anonymize http headers
> 
> forwarded_for off
> 
> request_header_access Allow allow all
> 
> request_header_access Authorization allow all
> 
> request_header_access WWW-Authenticate allow all
> 
> request_header_access Proxy-Authorization allow all
> 
> request_header_access Proxy-Authenticate allow all
> 
> request_header_access Cache-Control allow all
> 
> request_header_access Content-Encoding allow all
> 
> request_header_access Content-Length allow all
> 
> request_header_access Content-Type allow all
> 
> request_header_access Date allow all
> 
> request_header_access Expires allow all
> 
> request_header_access Host allow all
> 
> request_header_access If-Modified-Since allow all
> 
> request_header_access Last-Modified allow all
> 
> request_header_access Location allow all
> 
> request_header_access Pragma allow all
> 
> request_header_access Accept allow all
> 
> request_header_access Accept-Charset allow all
> 
> request_header_access Accept-Encoding allow all
> 
> request_header_access Accept-Language allow all
> 
> request_header_access Content-Language allow all
> 
> request_header_access Mime-Version allow all
> 
> request_header_access Retry-After allow all
> 
> request_header_access Title allow all
> 
> request_header_access Connection allow all
> 
> request_header_access Proxy-Connection allow all
> 
> request_header_access User-Agent allow all
> 
> request_header_access Cookie allow all
> 
> request_header_access All deny all
> 
> 
> 
> 
> 
> #
> 
> # Add any of your own refresh_pattern entries above these.
> 
> #
> 
> #refresh_pattern ^ftp:           1440    20%     10080
> 
> #refresh_pattern ^gopher:        1440    0%      1440
> 
> #refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
> 
> #refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
> 
> #refresh_pattern .               0       20%     4320
> 
> 
> ################################
> 
> acl me proxy_auth ye-1
> 
> cache_peer my.proxy.com <http://my.proxy.com/> parent 31280
> login=user1:password1 no-query name=a1
> 
> cache_peer_access a1 allow me
> 
> cache_peer_access a1 deny all
> 
> 
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>