[squid-users] Squid 6 with ssl-bump doesn't cache binary content over 100 kb
Alex Rousskov
rousskov at measurement-factory.com
Mon Feb 3 16:29:30 UTC 2025
On 2025-02-03 09:07, Thomas PALFRAY wrote:
> My team and I are working on setting up *a squid with ssl-bump* to cache
> binary content (jpeg, png, pdf and json) on a remote site over HTTPS.
>
> The size of the binary content can vary from a few dozen KB to several
> hundred MB.
>
> We had a working HTTP configuration under squid 3.14, but for security
> reasons, all our links must now go through HTTPS.
>
> We've tried configuring Squid 5 and Squid 6, but the behavior described
> below is the same for both versions of the tool.
>
> In fact, we're seeing 200/TCP_MISS_ABORTED codes for content larger than
> around 100kb.
There is not enough information to be sure, but:
* If your client sends large requests, then this could be Bug 5405 fixed
in v6.13.
* Otherwise, it could be Bug 5352 (also fixed in v6.13), even though the
known manifestation of that bug is related to ICAP content inspection
that your Squid does not use.
If you have not tested v6.13, please do.
HTH,
Alex.
> When content sizes are smaller, the expected behaviour occurs and data
> is returned from the cache.
>
> On the client side, for content larger than 100kb :
>
> * squid 5 returns x-cache = “MISS” and x-cache-lookup = “HIT”.
> * squid 6 returns cache-status = “url;detail=match”, which is
> equivalent if my understanding is correct.
>
> We first thought it might be linked to the following bug:
> _https://bugs.squid-cache.org/show_bug.cgi?id=5214
> <https://bugs.squid-cache.org/show_bug.cgi?id=5214>_, which encouraged
> us to try version 6, but without success.
>
> * Is it a configuration problem that we missed?
> * Can you help us ?
>
> HTTP client-side headers returned by squid 6 :
>
> //
>
> /[2025-01-14T08:46:49.875] [TRACE] default - [0/1] getContentStream
> (user) header: {/
>
> / "date": "Tue, 14 Jan 2025 05:47:30 GMT",/
>
> / "server": "Apache-Chemistry-OpenCMIS/1.2.0_1859862-XXXXX-1",/
>
> / "strict-transport-security": "max-age=15768000",/
>
> /"x-xss-protection": "1; mode=block",/
>
> ///"x-frame-options": "SAMEORIGIN",/
>
> / "x-content-type-options": "nosniff",/
>
> / "access-control-allow-origin": "*",/
>
> / "access-control-expose-headers": "Content-Disposition",/
>
> / "cache-control": "public, s-maxage=3600, must-revalidate",/
>
> / "etag":
> "\"08bd240128b475722db82d36c7ae7f164c37cab4ad2480abae052875fe7bc3bfdfef9996197d40110a13208d39a3db3a789879bc31803c82f25211eeba505455\"",/
>
> / "content-disposition": "inline; filename=FILENAME.jpg",/
>
> / "content-type": "image/jpeg",/
>
> / "content-length": "2069587",/
>
> / "content-security-policy": "default-src 'none'; connect-src 'self';
> font-src 'self' fonts.gstatic.com; img-src blob: 'self'; script-src
> 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline'
> fonts.googleapis.com",/
>
> */ "cache-status": "sv-infra-pxy4;detail=match",/*
>
> */"via": "1.1 sv-infra-pxy4 (squid/6.10)",/*
>
> ///"connection": "close"/
>
> /}/
>
> /[2025-01-14T08:46:49.875] [INFO] default - [0/1][2] getCS oId:'369926'
> sId:*'80669'*name:' filename= FILENAME.jpg' type:'image/jpeg'
> size:*'2021.08ko'*res:[via:true hit:false length:true]/
>
> //
>
> //
>
> squid 6 acces.log extract:
>
> //
>
> //
>
> //
>
> full server configuration file :
>
> //
>
> /acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network
> (LAN)/
>
> /acl localnet src 10.0.0.0/8 # RFC 1918 local private
> network (LAN)/
>
> /acl localnet src 100.64.0.0/10 # RFC 6598
> shared address space (CGN)/
>
> /acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly
> plugged) machines/
>
> /acl localnet src 172.16.0.0/12 # RFC 1918
> local private network (LAN)/
>
> /acl localnet src 192.168.0.0/16 # RFC 1918 local
> private network (LAN)/
>
> /acl localnet src fc00::/7 # RFC 4193 local private network range/
>
> /acl localnet src fe80::/10 # RFC 4291 link-local (directly
> plugged) machines/
>
> //
>
> /acl SSL_ports port 443/
>
> /acl Safe_ports port 80 # http/
>
> /acl Safe_ports port 21 # ftp/
>
> /acl Safe_ports port 443 # https/
>
> /acl Safe_ports port 70 # gopher/
>
> /acl Safe_ports port 210 # wais/
>
> /acl Safe_ports port 1025-65535 # unregistered ports/
>
> /acl Safe_ports port 280 # http-mgmt/
>
> /acl Safe_ports port 488 # gss-http/
>
> /acl Safe_ports port 591 # filemaker/
>
> /acl Safe_ports port 777 # multiling http/
>
> //
>
> /#ACL pour SmartGED/
>
> /acl smartged-mime req_mime_type -i ^image/jpeg$/
>
> /acl smartged-mime req_mime_type -i ^image/png$/
>
> /acl smartged-mime req_mime_type -i ^application/pdf$/
>
> /acl smartged-mime req_mime_type -i ^application/json$/
>
> //
>
> /acl intermediate_fetching transaction_initiator certificate-fetching/
>
> //
>
> /http_access deny !Safe_ports/
>
> /http_access deny CONNECT !SSL_ports/
>
> //
>
> /# Only allow cachemgr access from localhost/
>
> /http_access allow localhost manager/
>
> /http_access deny manager/
>
> /http_access allow localnet/
>
> /http_access deny all/
>
> //
>
> /logformat toto %{%d/%b/%Y:%H:%M:%S}tl.%tu %>a %rm %03Hs/%Ss %6tr %ru %mt/
>
> //
>
> /access_log /var/log/squid/access.log toto/
>
> //
>
> /http_port 3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=4MB tls-cert=/etc/squid/ssl_cert/xxxx_.pem
> tls-key=/etc/squid/ssl_cert/xxxx_.key
> tls-dh=/etc/squid/ssl_cert/dhparam.pem/
>
> /tls_outgoing_options
> cipher=ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384/
>
> /sslcrtd_program /usr/lib/squid/security_file_certgen -s
> /var/spool/squid/ssl_db -M 20MB/
>
> /sslcrtd_children 5/
>
> /ssl_bump server-first all/
>
> /ssl_bump splice all/
>
> /sslproxy_cert_error allow all/
>
> //
>
> /# ACL deny pour les fichiers javascript/
>
> /acl denyjs urlpath_regex \.js/
>
> /# ok deny all requests above/
>
> /cache deny denyjs/
>
> //
>
> /# Leave coredumps in the first cache dir/
>
> /coredump_dir /cache1/squid/
>
> //
>
> /# Affichage des requetes GET completes dans cache.log/
>
> /strip_query_terms off/
>
> //
>
> /cache_mem 20480 MB/
>
> /# début d'éviction du cache/
>
> /cache_swap_low 50/
>
> /# aggressivité maximal de l'algorithme d'éviction du cache/
>
> /cache_swap_high 80/
>
> /# The LRU policies keeps recently referenced objects./
>
> /cache_replacement_policy lru/
>
> //
>
> /#/
>
> /# Add any of your own refresh_pattern entries above these./
>
> /#/
>
> /refresh_pattern ^ftp: 1440 20% 10080/
>
> /refresh_pattern -i (/cgi-bin/|\?) 0 0% 0/
>
> /refresh_pattern . 0 20% 4320/
>
> //
>
> /ftp_user _anonymous at xxx.fr <mailto:anonymous at xxx.fr>_/
>
> /#acl QUERY urlpath_regex cgi-bin \?/
>
> /#no_cache deny QUERY/
>
> /client_request_buffer_max_size 10240 KB/
>
> /maximum_object_size 1024 MB/
>
> /# to keep object in memory cache/
>
> /maximum_object_size_in_memory 10240 KB/
>
> /#to cause Squid to prefetch the whole file/
>
> /range_offset_limit 16 MB/
>
> /#quick_abort_min -1/
>
> /#Définition de la taille maximum d'un en-tête HTTP lors d'une requête/
>
> /reply_header_max_size 8192 KB/
>
> /#taille maximum d'un en-tête de réponse HTTP/
>
> /reply_header_max_size 8192 KB/
>
> //
>
> /# Uncomment and adjust the following to add a disk cache directory./
>
> /cache_dir aufs /cache1/squid 1024000 16 256 max-size=16777216/
>
> //
>
> //
>
> /error_directory /usr/share/squid/errors/fr/
>
> /half_closed_clients off/
>
> /max_filedescriptors 8192/
>
> /forward_max_tries 50/
>
> //
>
> /#Affichage du store-id/
>
> /cache_store_log daemon:/var/log/squid/store_daemon.log/
>
> /cache_store_log stdio:/var/log/squid/store_stdio.log/
>
> /#debug_options "ALL,3 33,7 47,7 61,7 85,7"/
>
> /# Log all critical and important messages./
>
> /#debug_options ALL,1/
>
> /debug_options ALL,3/
>
> //
>
> /# Enable SQUID's SNMP/
>
> /snmp_port 3401/
>
> /acl snmppublic snmp_community PASdePUBLIC/
>
> /snmp_access allow snmppublic all/
>
> Best regards,
>
> *Thomas PALFRAY*
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> https://lists.squid-cache.org/listinfo/squid-users
More information about the squid-users
mailing list