[squid-users] Squid 6 with ssl-bump doesn't cache binary content over 100 kb
Alex Rousskov
rousskov at measurement-factory.com
Tue Feb 25 14:56:21 UTC 2025
On 2025-02-25 09:47, Thomas PALFRAY wrote:
> we tried version 6.13 as recommended, but the behavior is the same.
Thank you for testing v6.13. That test eliminates many suspects.
> What additional information would you need to understand the the problem
For the next step in triage, I can offer a free private review of your
cache.log file collected while reproducing the problem using as few
transactions as possible and enabling full debugging (e.g., setting
debug_options to ALL,9). More hints are available at
https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction
If you would like to proceed with the above analysis, please email me a
link to the corresponding compressed cache.log.
Thank you,
Alex.
> *De :*squid-users <squid-users-bounces at lists.squid-cache.org> *De la
> part de* Thomas PALFRAY
> *Envoyé :* lundi 3 février 2025 17:08
> *À :* squid-users at lists.squid-cache.org
> *Objet :* [squid-users] Squid 6 with ssl-bump doesn't cache binary
> content over 100 kb
>
> Hello everyone,
>
> My team and I are working on setting up *a squid with ssl-bump* to cache
> binary content (jpeg, png, pdf and json) on a remote site over HTTPS.
>
> The size of the binary content can vary from a few dozen KB to several
> hundred MB.
>
> We had a working HTTP configuration under squid 3.14, but for security
> reasons, all our links must now go through HTTPS.
>
> We've tried configuring Squid 5 and Squid 6, but the behavior described
> below is the same for both versions of the tool.
>
> In fact, we're seeing 200/TCP_MISS_ABORTED codes for content larger than
> around 100kb.
>
> When content sizes are smaller, the expected behaviour occurs and data
> is returned from the cache.
>
> On the client side, for content larger than 100kb :
>
> * squid 5 returns x-cache = “MISS” and x-cache-lookup = “HIT”.
> * squid 6 returns cache-status = “url;detail=match”, which is
> equivalent if my understanding is correct.
>
> We first thought it might be linked to the following bug:
> _https://bugs.squid-cache.org/show_bug.cgi?id=5214
> <https://bugs.squid-cache.org/show_bug.cgi?id=5214>_, which encouraged
> us to try version 6, but without success.
>
> * Is it a configuration problem that we missed?
> * Can you help us ?
>
> HTTP client-side headers returned by squid 6 :
>
> //
>
> /[2025-01-14T08:46:49.875] [TRACE] default - [0/1] getContentStream
> (user) header: {/
>
> / "date": "Tue, 14 Jan 2025 05:47:30 GMT",/
>
> / "server": "Apache-Chemistry-OpenCMIS/1.2.0_1859862-XXXXX-1",/
>
> / "strict-transport-security": "max-age=15768000",/
>
> /"x-xss-protection": "1; mode=block",/
>
> ///"x-frame-options": "SAMEORIGIN",/
>
> / "x-content-type-options": "nosniff",/
>
> / "access-control-allow-origin": "*",/
>
> / "access-control-expose-headers": "Content-Disposition",/
>
> / "cache-control": "public, s-maxage=3600, must-revalidate",/
>
> / "etag":
> "\"08bd240128b475722db82d36c7ae7f164c37cab4ad2480abae052875fe7bc3bfdfef9996197d40110a13208d39a3db3a789879bc31803c82f25211eeba505455\"",/
>
> / "content-disposition": "inline; filename=FILENAME.jpg",/
>
> / "content-type": "image/jpeg",/
>
> / "content-length": "2069587",/
>
> / "content-security-policy": "default-src 'none'; connect-src 'self';
> font-src 'self' fonts.gstatic.com; img-src blob: 'self'; script-src
> 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'self' 'unsafe-inline'
> fonts.googleapis.com",/
>
> */ "cache-status": "sv-infra-pxy4;detail=match",/*
>
> */"via": "1.1 sv-infra-pxy4 (squid/6.10)",/*
>
> ///"connection": "close"/
>
> /}/
>
> /[2025-01-14T08:46:49.875] [INFO] default - [0/1][2] getCS oId:'369926'
> sId:*'80669'*name:' filename= FILENAME.jpg' type:'image/jpeg'
> size:*'2021.08ko'*res:[via:true hit:false length:true]/
>
> //
>
> //
>
> squid 6 acces.log extract:
>
> //
>
> //
>
> //
>
> full server configuration file :
>
> //
>
> /acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network
> (LAN)/
>
> /acl localnet src 10.0.0.0/8 # RFC 1918 local private
> network (LAN)/
>
> /acl localnet src 100.64.0.0/10 # RFC 6598
> shared address space (CGN)/
>
> /acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly
> plugged) machines/
>
> /acl localnet src 172.16.0.0/12 # RFC 1918
> local private network (LAN)/
>
> /acl localnet src 192.168.0.0/16 # RFC 1918 local
> private network (LAN)/
>
> /acl localnet src fc00::/7 # RFC 4193 local private network range/
>
> /acl localnet src fe80::/10 # RFC 4291 link-local (directly
> plugged) machines/
>
> //
>
> /acl SSL_ports port 443/
>
> /acl Safe_ports port 80 # http/
>
> /acl Safe_ports port 21 # ftp/
>
> /acl Safe_ports port 443 # https/
>
> /acl Safe_ports port 70 # gopher/
>
> /acl Safe_ports port 210 # wais/
>
> /acl Safe_ports port 1025-65535 # unregistered ports/
>
> /acl Safe_ports port 280 # http-mgmt/
>
> /acl Safe_ports port 488 # gss-http/
>
> /acl Safe_ports port 591 # filemaker/
>
> /acl Safe_ports port 777 # multiling http/
>
> //
>
> /#ACL pour SmartGED/
>
> /acl smartged-mime req_mime_type -i ^image/jpeg$/
>
> /acl smartged-mime req_mime_type -i ^image/png$/
>
> /acl smartged-mime req_mime_type -i ^application/pdf$/
>
> /acl smartged-mime req_mime_type -i ^application/json$/
>
> //
>
> /acl intermediate_fetching transaction_initiator certificate-fetching/
>
> //
>
> /http_access deny !Safe_ports/
>
> /http_access deny CONNECT !SSL_ports/
>
> //
>
> /# Only allow cachemgr access from localhost/
>
> /http_access allow localhost manager/
>
> /http_access deny manager/
>
> /http_access allow localnet/
>
> /http_access deny all/
>
> //
>
> /logformat toto %{%d/%b/%Y:%H:%M:%S}tl.%tu %>a %rm %03Hs/%Ss %6tr %ru %mt/
>
> //
>
> /access_log /var/log/squid/access.log toto/
>
> //
>
> /http_port 3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=4MB tls-cert=/etc/squid/ssl_cert/xxxx_.pem
> tls-key=/etc/squid/ssl_cert/xxxx_.key
> tls-dh=/etc/squid/ssl_cert/dhparam.pem/
>
> /tls_outgoing_options
> cipher=ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384/
>
> /sslcrtd_program /usr/lib/squid/security_file_certgen -s
> /var/spool/squid/ssl_db -M 20MB/
>
> /sslcrtd_children 5/
>
> /ssl_bump server-first all/
>
> /ssl_bump splice all/
>
> /sslproxy_cert_error allow all/
>
> //
>
> /# ACL deny pour les fichiers javascript/
>
> /acl denyjs urlpath_regex \.js/
>
> /# ok deny all requests above/
>
> /cache deny denyjs/
>
> //
>
> /# Leave coredumps in the first cache dir/
>
> /coredump_dir /cache1/squid/
>
> //
>
> /# Affichage des requetes GET completes dans cache.log/
>
> /strip_query_terms off/
>
> //
>
> /cache_mem 20480 MB/
>
> /# début d'éviction du cache/
>
> /cache_swap_low 50/
>
> /# aggressivité maximal de l'algorithme d'éviction du cache/
>
> /cache_swap_high 80/
>
> /# The LRU policies keeps recently referenced objects./
>
> /cache_replacement_policy lru/
>
> //
>
> /#/
>
> /# Add any of your own refresh_pattern entries above these./
>
> /#/
>
> /refresh_pattern ^ftp: 1440 20% 10080/
>
> /refresh_pattern -i (/cgi-bin/|\?) 0 0% 0/
>
> /refresh_pattern . 0 20% 4320/
>
> //
>
> /ftp_user _anonymous at xxx.fr <mailto:anonymous at xxx.fr>_/
>
> /#acl QUERY urlpath_regex cgi-bin \?/
>
> /#no_cache deny QUERY/
>
> /client_request_buffer_max_size 10240 KB/
>
> /maximum_object_size 1024 MB/
>
> /# to keep object in memory cache/
>
> /maximum_object_size_in_memory 10240 KB/
>
> /#to cause Squid to prefetch the whole file/
>
> /range_offset_limit 16 MB/
>
> /#quick_abort_min -1/
>
> /#Définition de la taille maximum d'un en-tête HTTP lors d'une requête/
>
> /reply_header_max_size 8192 KB/
>
> /#taille maximum d'un en-tête de réponse HTTP/
>
> /reply_header_max_size 8192 KB/
>
> //
>
> /# Uncomment and adjust the following to add a disk cache directory./
>
> /cache_dir aufs /cache1/squid 1024000 16 256 max-size=16777216/
>
> //
>
> //
>
> /error_directory /usr/share/squid/errors/fr/
>
> /half_closed_clients off/
>
> /max_filedescriptors 8192/
>
> /forward_max_tries 50/
>
> //
>
> /#Affichage du store-id/
>
> /cache_store_log daemon:/var/log/squid/store_daemon.log/
>
> /cache_store_log stdio:/var/log/squid/store_stdio.log/
>
> /#debug_options "ALL,3 33,7 47,7 61,7 85,7"/
>
> /# Log all critical and important messages./
>
> /#debug_options ALL,1/
>
> /debug_options ALL,3/
>
> //
>
> /# Enable SQUID's SNMP/
>
> /snmp_port 3401/
>
> /acl snmppublic snmp_community PASdePUBLIC/
>
> /snmp_access allow snmppublic all/
>
> Best regards,
>
> *Thomas PALFRAY*
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> https://lists.squid-cache.org/listinfo/squid-users
More information about the squid-users
mailing list