[squid-users] Squid 6 with ssl-bump doesn't cache binary content over 100 kb
Thomas PALFRAY
tpal at numen.mg
Mon Feb 3 14:07:46 UTC 2025
Hello everyone,
My team and I are working on setting up a squid with ssl-bump to cache binary content (jpeg, png, pdf and json) on a remote site over HTTPS.
The size of the binary content can vary from a few dozen KB to several hundred MB.
We had a working HTTP configuration under squid 3.14, but for security reasons, all our links must now go through HTTPS.
We've tried configuring Squid 5 and Squid 6, but the behavior described below is the same for both versions of the tool.
In fact, we're seeing 200/TCP_MISS_ABORTED codes for content larger than around 100kb.
When content sizes are smaller, the expected behaviour occurs and data is returned from the cache.
On the client side, for content larger than 100kb :
* squid 5 returns x-cache = "MISS" and x-cache-lookup = "HIT".
* squid 6 returns cache-status = "url;detail=match", which is equivalent if my understanding is correct.
We first thought it might be linked to the following bug: https://bugs.squid-cache.org/show_bug.cgi?id=5214, which encouraged us to try version 6, but without success.
* Is it a configuration problem that we missed?
* Can you help us ?
HTTP client-side headers returned by squid 6 :
[2025-01-14T08:46:49.875] [TRACE] default - [0/1] getContentStream (user) header: {
"date": "Tue, 14 Jan 2025 05:47:30 GMT",
"server": "Apache-Chemistry-OpenCMIS/1.2.0_1859862-XXXXX-1",
"strict-transport-security": "max-age=15768000",
"x-xss-protection": "1; mode=block",
"x-frame-options": "SAMEORIGIN",
"x-content-type-options": "nosniff",
"access-control-allow-origin": "*",
"access-control-expose-headers": "Content-Disposition",
"cache-control": "public, s-maxage=3600, must-revalidate",
"etag": "\"08bd240128b475722db82d36c7ae7f164c37cab4ad2480abae052875fe7bc3bfdfef9996197d40110a13208d39a3db3a789879bc31803c82f25211eeba505455\"",
"content-disposition": "inline; filename=FILENAME.jpg",
"content-type": "image/jpeg",
"content-length": "2069587",
"content-security-policy": "default-src 'none'; connect-src 'self'; font-src 'self' fonts.gstatic.com; img-src blob: 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline' fonts.googleapis.com",
"cache-status": "sv-infra-pxy4;detail=match",
"via": "1.1 sv-infra-pxy4 (squid/6.10)",
"connection": "close"
}
[2025-01-14T08:46:49.875] [INFO] default - [0/1][2] getCS oId:'369926' sId:'80669' name:' filename= FILENAME.jpg' type:'image/jpeg' size:'2021.08ko' res:[via:true hit:false length:true]
squid 6 acces.log extract:
[cid:image001.png at 01DB765E.24C1B6C0]
full server configuration file :
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN)
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
#ACL pour SmartGED
acl smartged-mime req_mime_type -i ^image/jpeg$
acl smartged-mime req_mime_type -i ^image/png$
acl smartged-mime req_mime_type -i ^application/pdf$
acl smartged-mime req_mime_type -i ^application/json$
acl intermediate_fetching transaction_initiator certificate-fetching
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access deny all
logformat toto %{%d/%b/%Y:%H:%M:%S}tl.%tu %>a %rm %03Hs/%Ss %6tr %ru %mt
access_log /var/log/squid/access.log toto
http_port 3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB tls-cert=/etc/squid/ssl_cert/xxxx_.pem tls-key=/etc/squid/ssl_cert/xxxx_.key tls-dh=/etc/squid/ssl_cert/dhparam.pem
tls_outgoing_options cipher=ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384
sslcrtd_program /usr/lib/squid/security_file_certgen -s /var/spool/squid/ssl_db -M 20MB
sslcrtd_children 5
ssl_bump server-first all
ssl_bump splice all
sslproxy_cert_error allow all
# ACL deny pour les fichiers javascript
acl denyjs urlpath_regex \.js
# ok deny all requests above
cache deny denyjs
# Leave coredumps in the first cache dir
coredump_dir /cache1/squid
# Affichage des requetes GET completes dans cache.log
strip_query_terms off
cache_mem 20480 MB
# début d'éviction du cache
cache_swap_low 50
# aggressivité maximal de l'algorithme d'éviction du cache
cache_swap_high 80
# The LRU policies keeps recently referenced objects.
cache_replacement_policy lru
#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
ftp_user anonymous at xxx.fr<mailto:anonymous at xxx.fr>
#acl QUERY urlpath_regex cgi-bin \?
#no_cache deny QUERY
client_request_buffer_max_size 10240 KB
maximum_object_size 1024 MB
# to keep object in memory cache
maximum_object_size_in_memory 10240 KB
#to cause Squid to prefetch the whole file
range_offset_limit 16 MB
#quick_abort_min -1
#Définition de la taille maximum d'un en-tête HTTP lors d'une requête
reply_header_max_size 8192 KB
#taille maximum d'un en-tête de réponse HTTP
reply_header_max_size 8192 KB
# Uncomment and adjust the following to add a disk cache directory.
cache_dir aufs /cache1/squid 1024000 16 256 max-size=16777216
error_directory /usr/share/squid/errors/fr
half_closed_clients off
max_filedescriptors 8192
forward_max_tries 50
#Affichage du store-id
cache_store_log daemon:/var/log/squid/store_daemon.log
cache_store_log stdio:/var/log/squid/store_stdio.log
#debug_options "ALL,3 33,7 47,7 61,7 85,7"
# Log all critical and important messages.
#debug_options ALL,1
debug_options ALL,3
# Enable SQUID's SNMP
snmp_port 3401
acl snmppublic snmp_community PASdePUBLIC
snmp_access allow snmppublic all
Best regards,
Thomas PALFRAY
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20250203/5ebdda56/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 43281 bytes
Desc: image001.png
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20250203/5ebdda56/attachment-0001.png>
More information about the squid-users
mailing list