[squid-users] Cache Peers and traffic handling

[NgTech LTD]

Its not clear what is the factor for a specific cache peer selection.
This will affect any advice.
Is it only baesd on username?

Eliezer

בתאריך יום ד׳, 14 באפר׳ 2021, 9:29, מאת koshik moshik ‏<
koshikmoshik at gmail.com>:

> Thank you! Yes, it works fine with 5 peers. So, what would be the best
> solution to handle 5000 peers?
>
> On Mon, Apr 12, 2021 at 6:03 PM Alex Rousskov <
> rousskov at measurement-factory.com> wrote:
>
>> On 4/10/21 5:03 PM, koshik moshik wrote:
>>
>> > I am trying to run a Squid proxy Server witth about 5000 cache peers. I
>> > am running a dedicated server with 6 cores and 32GB RAM on Ubuntu 16.
>> >
>> >
>> > Could you tell me what else is needed / not needed in my squid.config? I
>> > am encountering a high CPU usage and would like to create a very
>> > efficient proxy server.
>>
>> IIRC, Squid code is not optimized for handling a large number of
>> cache_peers: Several cache peer selection steps involve linear searches.
>>
>> I do not know what exactly causes high CPU usage in your environment but
>> it could be those linear searches. You can test that (indirectly) by
>> decreasing the number of cache_peers from 5000 to, say, 5. That is a
>> weak test, of course, because other cache_peer-related overheads could
>> be to blame, but I would start there.
>>
>>
>> HTH,
>>
>> Alex.
>>
>>
>>
>> > Down below you can find my squid.config(I deleted the other cache_peer
>> > lines):
>> >
>> > -----------
>> >
>> > http_port 3128
>> >
>> > dns_v4_first on
>> >
>> > acl SSL_ports port 1-65535
>> >
>> > acl Safe_ports port 1-65535
>> >
>> > acl CONNECT method CONNECT
>> >
>> > http_access deny !Safe_ports
>> >
>> > http_access deny CONNECT !SSL_ports
>> >
>> > auth_param basic program /usr/lib/squid/basic_ncsa_auth
>> /etc/squid/.htpasswd
>> >
>> > auth_param basic children 5
>> >
>> > auth_param basic realm Squid Basic Authentication
>> >
>> > auth_param basic credentialsttl 5 hours
>> >
>> > acl password proxy_auth REQUIRED
>> >
>> > http_access allow password
>> >
>> > #http_access deny all
>> >
>> > cache allow all
>> >
>> > never_direct allow all
>> >
>> > ident_access deny all
>> >
>> >
>> >
>> >
>> >
>> > cache_mem 1 GB
>> >
>> > maximum_object_size_in_memory 16 MB
>> >
>> >
>> >
>> >
>> >
>> > # Leave coredumps in the first cache dir
>> >
>> > coredump_dir /var/spool/squid
>> >
>> >
>> > #Rules to anonymize http headers
>> >
>> > forwarded_for off
>> >
>> > request_header_access Allow allow all
>> >
>> > request_header_access Authorization allow all
>> >
>> > request_header_access WWW-Authenticate allow all
>> >
>> > request_header_access Proxy-Authorization allow all
>> >
>> > request_header_access Proxy-Authenticate allow all
>> >
>> > request_header_access Cache-Control allow all
>> >
>> > request_header_access Content-Encoding allow all
>> >
>> > request_header_access Content-Length allow all
>> >
>> > request_header_access Content-Type allow all
>> >
>> > request_header_access Date allow all
>> >
>> > request_header_access Expires allow all
>> >
>> > request_header_access Host allow all
>> >
>> > request_header_access If-Modified-Since allow all
>> >
>> > request_header_access Last-Modified allow all
>> >
>> > request_header_access Location allow all
>> >
>> > request_header_access Pragma allow all
>> >
>> > request_header_access Accept allow all
>> >
>> > request_header_access Accept-Charset allow all
>> >
>> > request_header_access Accept-Encoding allow all
>> >
>> > request_header_access Accept-Language allow all
>> >
>> > request_header_access Content-Language allow all
>> >
>> > request_header_access Mime-Version allow all
>> >
>> > request_header_access Retry-After allow all
>> >
>> > request_header_access Title allow all
>> >
>> > request_header_access Connection allow all
>> >
>> > request_header_access Proxy-Connection allow all
>> >
>> > request_header_access User-Agent allow all
>> >
>> > request_header_access Cookie allow all
>> >
>> > request_header_access All deny all
>> >
>> >
>> >
>> >
>> >
>> > #
>> >
>> > # Add any of your own refresh_pattern entries above these.
>> >
>> > #
>> >
>> > #refresh_pattern ^ftp:           1440    20%     10080
>> >
>> > #refresh_pattern ^gopher:        1440    0%      1440
>> >
>> > #refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
>> >
>> > #refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
>> >
>> > #refresh_pattern .               0       20%     4320
>> >
>> >
>> > ################################
>> >
>> > acl me proxy_auth ye-1
>> >
>> > cache_peer my.proxy.com <http://my.proxy.com/> parent 31280
>> > login=user1:password1 no-query name=a1
>> >
>> > cache_peer_access a1 allow me
>> >
>> > cache_peer_access a1 deny all
>> >
>> >
>> > _______________________________________________
>> > squid-users mailing list
>> > squid-users at lists.squid-cache.org
>> > http://lists.squid-cache.org/listinfo/squid-users
>> >
>>
>> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210414/49312bf6/attachment-0001.htm>