[squid-users] Squid Reverse Proxy and WebDAV caching

[Amos Jeffries]

On 25/08/17 20:18, Olivier MARCHETTA wrote:
> Hello Amos,
> 
> Thank you for your help.
> I have probably misconfigured the refresh_pattern in my config file.
> Below more information.
> My squid conf file:
> 
> ---------------------------------------------------------------------
> http_port 10.10.10.10:3128

You said this was a reverse-proxy. This config file is for a 
forward/explicit proxy.

A reverse-proxy with the role you stated earlier would be configured with:

   http_port 3128
   http_port 80 accel
   https_port 443 accel cert=.. key=...
   cache_peer tenant.sharepoint.com parent 80 0 originserver
   acl SP dstdomain tenant.sharepoint.com
   cache_peer_access tenant.sharepoint.com allow SP
   http_access allow SP


> icp_port 0
> digest_generation off
> dns_v4_first on
> pid_filename /var/run/squid/squid.pid
> cache_effective_user squid
> cache_effective_group proxy
> error_default_language en
> icon_directory /usr/local/etc/squid/icons
> visible_hostname pfSense Firewall

As the name of the directive above indicates it is supposed to be a 
*hostname*. More specifically it is the publicly visible FQDN of the 
Squid server. It will be used in error pages URLs for fetching the icons 
etc.

"http://pfsense Firewall/" is a pretty funny URL for Squid.



> cache_mgr pfsense at mycomp.cloud
> access_log /var/squid/logs/access.log
> cache_log /var/squid/logs/cache.log
> cache_store_log none
> netdb_filename /var/squid/logs/netdb.state
> pinger_enable on
> pinger_program /usr/local/libexec/squid/pinger
> 
> logfile_rotate 7
> debug_options rotate=7
> shutdown_lifetime 3 seconds
> # Allow local network(s) on interface(s)
> acl localnet src  10.10.10.0/24
> forwarded_for on
> uri_whitespace strip
> 
> cache_mem 128 MB
> maximum_object_size_in_memory 20 MB
> memory_replacement_policy heap GDSF
> cache_replacement_policy heap LFUDA
> minimum_object_size 0 KB
> maximum_object_size 20 MB
> cache_dir ufs /var/squid/cache 300 16 256
> offline_mode on
> cache_swap_low 90
> cache_swap_high 95
> cache allow all

NP: its pretty pointless to configure things to their default values. 
You can simplify your config quite a lot by removing many of the above 
lines.

> # Add any of your own refresh_pattern entries above these.

Please re-read the above sentence from your squid.conf.

Order is important. <https://wiki.squid-cache.org/SquidFaq/OrderIsImportant>

> refresh_pattern ^ftp:    1440  20%  10080
> refresh_pattern ^gopher:  1440  0%  1440
> refresh_pattern -i (/cgi-bin/|\?) 0  0%  0
> refresh_pattern .    0  20%  4320
> refresh_pattern -i \.jpg$ 30 50% 4320 ignore-reload ignore-no-cache ignore-no-store ignore-private
> refresh_pattern -i \.pdf$ 30 50% 4320 ignore-reload ignore-no-cache ignore-no-store ignore-private
> refresh_pattern -i \.docx$ 30 50% 4320 ignore-reload ignore-no-cache ignore-no-store ignore-private


Also,

> 
> #Remote proxies
> 
> # Setup some default acls
> # ACLs all, manager, localhost, and to_localhost are predefined.
> acl allsrc src all

I suggest you double-check anywhere you are using the "allsrc" ACL. If 
it is not explicitly being used as a name to attach a deny_info to then 
it is a pointless waste of memory to redefine like this - just use the 
built-in 'all' ACL name.


> acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 4443 3128 3129 1025-65535

NP: with the 1025-65535 set of ports listed you don't need to have 
explicit entries for those ports higher than 1025.

Also, since this was apparently a reverse-proxy for HTTP and the log 
seems to show HTTPS as well - it will not be receiving any of those 
ports on URLs other than 80 and 443.


> acl sslports port 443 563 4443
> ---------------------------------------------------------------------
> 
> 
> The Squid access log:
> ---------------------------------------------------------------------
> Date   IP   Status   Address   User   Destination
> 24.08.2017 12:42:18   10.10.10.100   TCP_MISS/200   https://tenant.sharepoint.com/sites/Marketing/Shared%20Documents/picture.jpg
> 24.08.2017 12:42:17   10.10.10.100   TCP_MISS/200   https://tenant.sharepoint.com/sites/Marketing/Shared%20Documents/large1.pdf
> 24.08.2017 12:42:16   10.10.10.100   TCP_MISS/200   https://tenant.sharepoint.com/sites/Marketing/Shared%20Documents/large1.docx
> ---------------------------------------------------------------------
> 
> 
> The cache manager info:
> ---------------------------------------------------------------------
> Cache information for squid:
>     Hits as % of all requests:   5min: 0.0%, 60min: 0.0%
>     Hits as % of bytes sent:   5min: 0.0%, 60min: 0.0%
>     Memory hits as % of hit requests:   5min: 0.0%, 60min: 0.0%
>     Disk hits as % of hit requests:   5min: 0.0%, 60min: 0.0%
>     Storage Swap size:   0 KB
>     Storage Swap capacity:    0.0% used, 100.0% free
>     Storage Mem size:   216 KB
>     Storage Mem capacity:    0.2% used, 99.8% free
>     Mean Object Size:   0.00 KB
> ---------------------------------------------------------------------
> 

Okay, not much caching. You got that debug trace?

Amos