[squid-users] My cache gived me a content-length of 0, and a 200 TCP_REFRESH_UNMODIFIED_ABORTED

Wed Feb 24 15:45:43 UTC 2021

Thanks Amos - that’s a very kind and thorough explanation.

Ok - I understand why the content-length is 0 as the server responded with a 304. Gotcha. Squid in response responds with a 200.

And I also understand why it is a REFRESH because the server responded saying that it should not cache. (As a side note - I don’t understand what these directives do then: ignore-no-cache ignore-no-store ignore-must-revalidate ignore-private)

What isn’t clear to me then is whether Squid actually returns the cached image? (content-length being 0).

The reason being, is that I am using Squid between my server and AWS S3 (or Google Storage)

The first time I call my nodejs code to fetch the image from S3 (via the proxy) it works.
The second time I call my nodejs code to fetch the image from S3, it fails. (Same Squid logs - instead of curl I am using nodejs code)

My nodejs code fails with an HTTP Parse error (HPE_INVALID_CONSTANT). My guess is that the response from Squid  is malformed. My guess is that the response is empty. Or that the image is in the response, but the content-length 0 causes the nodejs HTTP response parser to fail.

I don’t understand why my code behaves differently when it is receiving the image for the first time, and when it is receiving the cached image.

> On 24 Feb 2021, at 13:50, Señor J Onion <senor.j.onion at gmail.com> wrote:
> 
> I am new to Squid, I have been trying to get this to work for almost two weeks now, and have found nothing in the archives.
> 
> This is my curl command (you will get a 403 forbidden by the time you run this dear reader):
> 
> curl -s -D - -o /dev/null -G -d "alt=media" -x "http://localhost:3128" http://storage.googleapis.com/omgimg.appspot.com/test.jpeg -H "host:storage.googleapis.com" -H "x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" -H "x-amz-date:20210224T111631Z" -H "authorization:AWS4-HMAC-SHA256 Credential=GOOG1EGG4VCQ2EVRCJ2JCIO7ZDSZ3CY45Q72ATYZU2P32HITBFUOVQ6TEBWXI/20210224/auto/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=77a60480e47dda2b65ef3ebcd72a032458685e74e2560bb9083dbb03c3f6c13d”
> 
> 
> These are the HTTP response headers:
> 
> 
> FIRST RUN:
> 
> HTTP/1.1 200 OK
> X-GUploader-UploadID: ABg5-UwzmWjdWRPkonOxhLl3rUUik6wN3MB_ME1w1pxS5Rtmp_Cl1AAiP5G3tA9oXpFfAMnLCn5Pb8VW1mioc6GI-wJDun1S_g
> Expires: Wed, 24 Feb 2021 11:20:08 GMT
> Date: Wed, 24 Feb 2021 11:20:08 GMT
> Cache-Control: private, max-age=0
> Last-Modified: Tue, 04 Aug 2020 12:09:00 GMT
> ETag: "d5b65c332fb6f80a0eade692b40b4afd"
> Content-Type: image/jpeg
> x-goog-hash: crc32c=6ijxaQ==
> x-goog-hash: md5=1bZcMy+2+AoOreaStAtK/Q==
> x-goog-storage-class: STANDARD
> Accept-Ranges: bytes
> Content-Length: 2296040
> Server: UploadServer
> X-Cache: MISS from 80396e157a13
> X-Cache-Lookup: MISS from 80396e157a13:3128
> Via: 1.1 80396e157a13 (squid/3.5.27)
> Connection: keep-alive
> 
> Squid log: 200 2296721 TCP_MISS:HIER_DIRECT
> 
> 
> SECOND RUN:
> 
> HTTP/1.1 200 OK
> Content-Type: image/jpeg
> x-goog-hash: crc32c=6ijxaQ==
> x-goog-hash: md5=1bZcMy+2+AoOreaStAtK/Q==
> x-goog-storage-class: STANDARD
> Accept-Ranges: bytes
> X-GUploader-UploadID: ABg5-UzooDZGnVTXxGIWQ2i25EasnR3glFz41FfUFvclACmZb3iDccpSXsGbRH0cr-8lofOc6Wb3knUzYMTgj_wdrzo
> Expires: Wed, 24 Feb 2021 11:20:17 GMT
> Date: Wed, 24 Feb 2021 11:20:17 GMT
> Cache-Control: private, max-age=0
> Last-Modified: Tue, 04 Aug 2020 12:09:00 GMT
> ETag: "d5b65c332fb6f80a0eade692b40b4afd"
> Content-Length: 0
> Server: UploadServer
> Age: 0
> X-Cache: HIT from 80396e157a13
> X-Cache-Lookup: HIT from 80396e157a13:3128
> Via: 1.1 80396e157a13 (squid/3.5.27)
> Connection: keep-alive
> 
> Squid log: 200 651397 TCP_REFRESH_UNMODIFIED_ABORTED:HIER_DIRECT
> 
> 
> I don’t know why the second time round, I a) don’t get a HIT, and b) why my content-length is 0.
> 
> I think - this is the reason why - when I make these same calls in nodejs I end up with an HTTP Parse error (HPE_INVALID_CONSTANT).
> 
> 
> My squid.conf file looks like this:
> 
> acl localnet src 0.0.0.1-0.255.255.255
> acl localnet src 10.0.0.0/8
> acl localnet src 100.64.0.0/10
> acl localnet src 169.254.0.0/16
> acl localnet src 172.16.0.0/12
> acl localnet src 192.168.0.0/16
> acl localnet src fc00::/7
> acl localnet src fe80::/10
> 
> acl SSL_ports port 443
> acl Safe_ports port 80
> acl Safe_ports port 443
> 
> acl Connect_ports port 80
> acl Connect_ports port 443
> 
> acl CONNECT method CONNECT
> 
> http_access deny !Safe_ports
> 
> http_access deny CONNECT !Connect_ports
> 
> http_access allow localhost manager
> http_access deny manager
> 
> http_access deny to_localhost
> 
> http_access allow localnet
> http_access allow localhost
> 
> http_access deny all
> 
> strip_query_terms off
> refresh_pattern . 525600 100% 525600 override-expire override-lastmod ignore-reload ignore-no-cache ignore-no-store reload-into-ims ignore-must-revalidate ignore-private ignore-auth store-stale
> 
> cache_mem 2500 MB
> maximum_object_size_in_memory 100 MB
> memory_cache_mode always
> 
> 
> 
> 
> Any direction would be greatly appreciated!
> 
> 
> 
> 