[squid-users] Squid Reverse Proxy and WebDAV caching

Fri Aug 25 08:18:05 UTC 2017

Hello Amos,

Thank you for your help.
I have probably misconfigured the refresh_pattern in my config file.
Below more information.
My squid conf file:

---------------------------------------------------------------------
http_port 10.10.10.10:3128
icp_port 0
digest_generation off
dns_v4_first on
pid_filename /var/run/squid/squid.pid
cache_effective_user squid
cache_effective_group proxy
error_default_language en
icon_directory /usr/local/etc/squid/icons
visible_hostname pfSense Firewall
cache_mgr pfsense at mycomp.cloud
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
netdb_filename /var/squid/logs/netdb.state
pinger_enable on
pinger_program /usr/local/libexec/squid/pinger

logfile_rotate 7
debug_options rotate=7
shutdown_lifetime 3 seconds
# Allow local network(s) on interface(s)
acl localnet src  10.10.10.0/24
forwarded_for on
uri_whitespace strip

cache_mem 128 MB
maximum_object_size_in_memory 20 MB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
minimum_object_size 0 KB
maximum_object_size 20 MB
cache_dir ufs /var/squid/cache 300 16 256
offline_mode on
cache_swap_low 90
cache_swap_high 95
cache allow all
# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:    1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0  0%  0
refresh_pattern .    0  20%  4320
refresh_pattern -i \.jpg$ 30 50% 4320 ignore-reload ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.pdf$ 30 50% 4320 ignore-reload ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.docx$ 30 50% 4320 ignore-reload ignore-no-cache ignore-no-store ignore-private

#Remote proxies

# Setup some default acls
# ACLs all, manager, localhost, and to_localhost are predefined.
acl allsrc src all
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 4443 3128 3129 1025-65535
acl sslports port 443 563 4443
---------------------------------------------------------------------

The Squid access log:
---------------------------------------------------------------------
Date   IP   Status   Address   User   Destination
24.08.2017 12:42:18   10.10.10.100   TCP_MISS/200   https://tenant.sharepoint.com/sites/Marketing/Shared%20Documents/picture.jpg
24.08.2017 12:42:17   10.10.10.100   TCP_MISS/200   https://tenant.sharepoint.com/sites/Marketing/Shared%20Documents/large1.pdf
24.08.2017 12:42:16   10.10.10.100   TCP_MISS/200   https://tenant.sharepoint.com/sites/Marketing/Shared%20Documents/large1.docx
---------------------------------------------------------------------

The cache manager info:
---------------------------------------------------------------------
Cache information for squid:
   Hits as % of all requests:   5min: 0.0%, 60min: 0.0%
   Hits as % of bytes sent:   5min: 0.0%, 60min: 0.0%
   Memory hits as % of hit requests:   5min: 0.0%, 60min: 0.0%
   Disk hits as % of hit requests:   5min: 0.0%, 60min: 0.0%
   Storage Swap size:   0 KB
   Storage Swap capacity:    0.0% used, 100.0% free
   Storage Mem size:   216 KB
   Storage Mem capacity:    0.2% used, 99.8% free
   Mean Object Size:   0.00 KB
---------------------------------------------------------------------

Regards,
Olivier MARCHETTA

-----Original Message-----
From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Amos Jeffries
Sent: Thursday, August 24, 2017 11:59 PM
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Squid Reverse Proxy and WebDAV caching

On 25/08/17 04:16, Olivier MARCHETTA wrote:
> Hello Squid Users,
> 
> I have configured a squid reverse proxy to access Microsoft SharePoint 
> Online with the aim of caching the document libraries into the squid 
> cache for a branch office.
> 
> But so far I can see the access log with the GET HTTP requests from 
> the users but none will be stored into the cache.
> 
> Now there are several difficulties to cache the documents:
> 
>  1. Microsoft is using SSL (but I have configured SSL bumps)  2. Files 
> are tagged with the cache header no-cache or cache-private

'no-cache' actually means things *are* cacheable. Squid just has to perform a quick check with the server before using them. Your logs should contain REFRESH instead of HIT entries for these objects.

The 'private' objects are only usable for one client, so caching is not useful. Latest Squid can cache them by configuring refresh_pattern directive ignore-private. Then Squid will do the REFRESH for these as well.

Welcome to HTTP/1.1 where things can be neither HIT nor MISS. The REFRESH means a server was involved, but the object delivered to the client may be new or from cache and of vastly different size than the refresh objects on the server connection.

IMPORTANT: do not configure ignore-private and ignore-must-revalidate for the same objects. That will corrupt your proxies responses.

>  3. The WebDAV client is the Microsoft Windows 10 client.
> 
> Now I would like to know if it’s still doable or if I can just forget 
> having this kind of configuration on squid, and move on to an alternate 
> caching method (OneDrive sync client for example).
> 

If you have a current up-to-date Squid it is probably caching but 
absence of the classical "HIT" tag being confusing.

If you are actively seeing MISS in the logs for these objects then we 
will need the HTTP transaction headers to see what is going on. That can 
be retrieved with a debug_options 11,2 trace.

Amos
_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users