[squid-users] Squid3: 100 % CPU load during object caching

[Jens Offenbach]

I have not tested rock yet, but I will give it a try and will report the results as soon as possible.

I have tested Squid 3.5.6 in three different virtualized environments (VMware vSphere, OpenStack Icehouse, OpenStack Kilo).

For clarification:
The 100 % CPU usage and the low download speed occurs only when there is NO cache hit and when the object is added to disk cache FOR THE FIRST TIME. When the object is in the cache, the download speed and the CPU usage of Squid (3.3.8 and 3.5.6) is optimal. I have tested it with an Ubuntu ISO image (>= 1 GB).

This is my current squid.config.

# ACCESS CONTROLS
# -----------------------------------------------------------------------------
  acl localnet    src 139.2.0.0/16
  acl localnet    src 193.96.112.0/21
  acl localnet    src 192.109.216.0/24
  acl localnet    src 100.1.4.0/22
  acl localnet    src 10.0.0.0/8
  acl localnet    src 172.16.0.0/12
  acl localnet    src 192.168.0.0/16
  http_access allow manager localhost
  http_access deny  manager
  http_access allow localnet
  http_access allow localhost
  http_access deny all

# NETWORK OPTIONS
# -----------------------------------------------------------------------------
  http_port 0.0.0.0:3128

# MEMORY CACHE OPTIONS
# -----------------------------------------------------------------------------
  maximum_object_size_in_memory 128 MB
  memory_replacement_policy heap LFUDA
  cache_mem 4 GB

# DISK CACHE OPTIONS
# -----------------------------------------------------------------------------
  maximum_object_size 10 GB
  cache_replacement_policy heap GDSF
  cache_dir aufs /var/cache/squid3 88894 16 256 max-size=10737418240

# LOGFILE OPTIONS
# -----------------------------------------------------------------------------
  access_log daemon:/var/log/squid3/access.log squid
  cache_store_log daemon:/var/log/squid3/store.log

# OPTIONS FOR TROUBLESHOOTING
# -----------------------------------------------------------------------------
  cache_log /var/log/squid3/cache.log
  coredump_dir /var/log/squid3
  
# OPTIONS FOR TUNING THE CACHE
# -----------------------------------------------------------------------------
  cache allow all
  
# OPTIONS INFLUENCING REQUEST FORWARDING 
# -----------------------------------------------------------------------------
  always_direct allow all

# MISCELLANEOUS
# -----------------------------------------------------------------------------
  memory_pools off


Thank you very much for your help! It is good to know that you will take care of my problem.

Regards,
Jens
 

Gesendet: Mittwoch, 22. Juli 2015 um 21:47 Uhr
Von: "Eliezer Croitoru" <eliezer at ngtech.co.il>
An: squid-users at lists.squid-cache.org
Cc: "Amos Jeffries" <squid3 at treenet.co.nz>
Betreff: Re: [squid-users] Squid3: 100 % CPU load during object caching
On 22/07/2015 21:59, Eliezer Croitoru wrote:
> Hey Jens,
>
> I have tested the issue with LARGE ROCK and not AUFS or UFS.
> Using squid or not my connection to the server is about 2.5 MBps (20Mbps).
> Squid is sitting on an intel atom with SSD drive and on a HIT case the
> download speed is more then doubled to 4.5 MBps(36Mbps).
> I have not tried it with AUFS yet.


And I must admit that AUFS beats rock cache with speed.
I have tried rock with basic "cache_dir rock /var/spool/squid 8000" vs
"cache_dir aufs /var/spool/squid 8000 16 256" and the aufs cache HIT
results more then doubles 3 the speed rock gave with default settings.

So about 15MBps which is 120Mbps.
I do not seem to feel what Jens feels but the 100% CPU might be because
of spinning disk hangs while reading the file from disk.

Amos, I remember that there were some suggestions how to tune large rock.
Any hints?
I can test it and make it a suggestion for big files.

Eliezer