[squid-users] offline mode not working for me

[Robin Carlisle]

Hi, Hoping someone can help me with this issue that I have been struggling
with for days now.   I am setting up squid on an ubuntu PC to forward HTTPS
requests to an API and an s3 bucket under my control on amazon AWS.  The
reason I am setting up the proxy is two-fold...

1) To reduce costs from AWS.
2) To provide content to the client on the ubuntu PC if there is a
networking issue somewhere in between the ubuntu PC and AWS.

Item 1 is going well so far.   Item 2 is not going well.   Setup details ...

*# squid - setup cache folder*
mkdir -p /var/cache/squid
chown -R proxy:proxy  /var/cache/squid

*# ssl - generate key*
apt --yes install squid-openssl libnss3-tools
openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 \
  -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=www.example.com" \
  -keyout /etc/squid/stuff.pem -out /etc/squid/stuff.pem
chown root:proxy /etc/squid/stuff.pem
chmod 644  /etc/squid/stuff.pem

*# ssl - ssl DB*
mkdir -p /var/lib/squid
rm -rf /var/lib/squid/ssl_db
/usr/lib/squid/security_file_certgen -c -s /var/lib/squid/ssl_db -M 4MB
chown -R proxy:proxy /var/lib/squid/ssl_db

*# /etc/squid/squid.conf :*
acl to_aws dstdomain .amazonaws.com
acl from_local src localhost
http_access allow to_aws
http_access allow from_local
cache allow all
cache_dir ufs /var/cache/squid 1024 16 256
offline_mode on
http_port 3129 ssl-bump cert=/etc/squid/stuff.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/lib/squid/security_file_certgen -s
/var/lib/squid/ssl_db -M 4MB
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all
sslproxy_cert_error deny all
cache_store_log stdio:/var/log/squid/store.log
logfile_rotate 0

*# /usr/bin/proxy-test :*
#!/bin/bash
curl --proxy http://localhost:3129 \
  --cacert /etc/squid/stuff.pem \
  -v "https://stuff.amazonaws.com/api/v1/stuff/stuff.json" \
  -H "Authorization: token MYTOKEN" \
  -H "Content-Type: application/json" \
  --output "/tmp/stuff.json"



When network connectivity is GOOD, everything works well and I get cache
HITS ...

*# /var/log/squid/access.log*
1705587538.837    238 127.0.0.1 NONE_NONE/200 0 CONNECT
stuff.amazonaws.com:443 - HIER_DIRECT/3.136.246.238 -
1705587538.838      0 127.0.0.1 TCP_MEM_HIT/200 32818 GET
https://stuff.amazonaws.com/api/v1/stuff/stuff.json - HIER_NONE/-
application/json

*# extract from /usr/bin/proxy-test output*
< HTTP/1.1 200 OK
< Date: Thu, 18 Jan 2024 13:38:01 GMT
< Content-Type: application/json
< Content-Length: 32187
< x-amzn-RequestId: 8afba80e-6df7-4d5b-a34b-a70bd9b54380
< Last-Modified: 2024-01-03T11:23:19.000Z
< Access-Control-Allow-Origin: *
< x-amz-apigw-id: RvN1CF2_iYcEokA=
< Cache-Control: max-age=2147483648,public,stale-if-error
< ETag: "53896156c4e8e26933188a092c4e40f1"
< X-Amzn-Trace-Id: Root=1-65a929b9-3bd3285934151c1a2495481a
< Age: 2578
< Warning: 110 squid/5.7 "Response is stale"
< X-Cache: HIT from ubuntu-pc
< X-Cache-Lookup: HIT from ubuntu-pc:3129
< Via: 1.1 ubuntu-pc (squid/5.7)
< Connection: keep-alive


When network connectivity is BAD, I get errors and a cache MISS.   In this
test case I unplugged the ethernet cable from the back on the ubuntu-pc ...

*# /var/log/squid/access.log*
1705588717.420     11 127.0.0.1 NONE_NONE/200 0 CONNECT
stuff.amazonaws.com:443 - HIER_DIRECT/3.135.162.228 -
1705588717.420      0 127.0.0.1 NONE_NONE/503 4087 GET
https://stuff.amazonaws.com/api/v1/stuff/stuff.json - HIER_NONE/- text/html

*# extract from /usr/bin/proxy-test output*
< HTTP/1.1 503 Service Unavailable
< Server: squid/5.7
< Mime-Version: 1.0
< Date: Thu, 18 Jan 2024 14:38:37 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3692
< X-Squid-Error: ERR_CONNECT_FAIL 101
< Vary: Accept-Language
< Content-Language: en
< X-Cache: MISS from ubuntu-pc
< X-Cache-Lookup: NONE from ubuntu-pc:3129
< Via: 1.1 ubuntu-pc (squid/5.7)
< Connection: close

I have also seen it error in a different way with a 502 but with the same
ultimate result.

My expectation/hope is that squid would return the cached object on any
network failure in between ubuntu-pc and the AWS endpoint - and continue to
return this cached object forever.   Is this something squid can do?   It
would seem that offline_mode should do this?

Hope you can help,

Robin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20240118/ac10bf0d/attachment.htm>