[squid-users] Squid3: 100 % CPU load during object caching

[Jens Offenbach]

I am running Squid3 3.3.8 on Ubuntu 14.04. Squid3 has been installed from the Ubuntu package repository. In my scenario, Squid has to cache big files >= 1 GB. At the moment, I am getting very bad transfer rates lower that 1 MB/sec. I have checked the connectivity using iperf3. It gives my a bandwith of 853 Mbits/sec between the nodes.

I have tried to investigate the problem and recognized that when there is no cache hit for a requested object, the Squid process reaches shortly after startup 100 % of one CPU core. The download rate drops down to 1 MB/sec. When I have a cache hit, I only get 30 MB/sec in my download.

Is there someting wrong with my config? I have already used Squid 3.3.14. I get the same result. Unfortunately, I was not able to build Squid 3.5.5 and 3.5.6.

Here is my squid.conf:
# ACCESS CONTROLS
# ----------------------------------------------------------------------------
  acl intranet    src 139.2.0.0/16
  acl intranet    src 193.96.112.0/21
  acl intranet    src 192.109.216.0/24
  acl intranet    src 100.1.4.0/22
  acl localnet    src 10.0.0.0/8
  acl localnet    src 172.16.0.0/12
  acl localnet    src 192.168.0.0/16
  acl localnet    src fc00::/7
  acl localnet    src fe80::/10
  acl to_intranet dst 139.2.0.0/16
  acl to_intranet dst 193.96.112.0/21
  acl to_intranet dst 192.109.216.0/24
  acl to_intranet dst 100.1.4.0/22
  acl to_localnet dst 10.0.0.0/8
  acl to_localnet dst 172.16.0.0/12
  acl to_localnet dst 192.168.0.0/16
  acl to_localnet dst fc00::/7
  acl to_localnet dst fe80::/10
  http_access allow manager localhost
  http_access deny  manager
  http_access allow localnet
  http_access allow localhost
  http_access deny all

# NETWORK OPTIONS
# ----------------------------------------------------------------------------
  http_port 0.0.0.0:3128

# OPTIONS WHICH AFFECT THE NEIGHBOR SELECTION ALGORITHM
# ----------------------------------------------------------------------------
  cache_peer proxy.mycompany.de parent 8080 0 no-query no-digest

# MEMORY CACHE OPTIONS
# ----------------------------------------------------------------------------
  maximum_object_size_in_memory 1 GB
  memory_replacement_policy heap LFUDA
  cache_mem 4 GB

# DISK CACHE OPTIONS
# ----------------------------------------------------------------------------
  maximum_object_size 10 GB
  cache_replacement_policy heap GDSF
  cache_dir aufs /var/cache/squid3 88894 16 256 max-size=10737418240

# LOGFILE OPTIONS
# ----------------------------------------------------------------------------
  access_log daemon:/var/log/squid3/access.log squid
  cache_store_log daemon:/var/log/squid3/store.log

# OPTIONS FOR TROUBLESHOOTING
# ----------------------------------------------------------------------------
  cache_log /var/log/squid3/cache.log
  coredump_dir /var/log/squid3

# OPTIONS FOR TUNING THE CACHE
# ----------------------------------------------------------------------------
  cache allow localnet
  cache allow localhost
  cache allow intranet
  cache deny  all
  refresh_pattern ^ftp:              1440    20%    10080
  refresh_pattern ^gopher:           1440     0%     1440
  refresh_pattern -i (/cgi-bin/|\?)     0     0%        0
  refresh_pattern .                     0    20%     4320

# HTTP OPTIONS
# ----------------------------------------------------------------------------
  via off

# ADMINISTRATIVE PARAMETERS
# ----------------------------------------------------------------------------
  cache_effective_user proxy
  cache_effective_group proxy

# ICP OPTIONS
# ----------------------------------------------------------------------------
  icp_port 0

# OPTIONS INFLUENCING REQUEST FORWARDING 
# ----------------------------------------------------------------------------
  nonhierarchical_direct on
  prefer_direct off
  always_direct allow to_localnet
  always_direct allow to_localhost
  always_direct allow to_intranet
  never_direct  allow all

# MISCELLANEOUS
# ----------------------------------------------------------------------------
  memory_pools off
  forwarded_for off