[squid-users] My cache gived me a content-length of 0, and a 200 TCP_REFRESH_UNMODIFIED_ABORTED

Thu Feb 25 17:49:55 UTC 2021

On 2/24/21 10:45 AM, Señor J Onion wrote:

> I don’t understand why my code behaves differently when it is
> receiving the image for the first time, and when it is receiving the
> cached image.

What you see is a result of two bugs.

* An origin server bug: During the second transaction, when Squid asks
the server whether there are any updates for the object cached during
the first transaction, the origin server lies to Squid about the changed
size of the object. The origin server gives Squid two contradictory
statements: "the object you cached has not changed at all (its strong
ETag remains the same)" and "the size of the object you cached has
changed (to zero)".

* A Squid bug or deficiency: Squid should detect an inconsistent server
response and, instead of serving the cached object with lying headers,
Squid should remove the cached object and request a fresh one (a cache
miss due to revalidation failure). Optionally, as an admin-authorized
"optimization", Squid can ignore the Content-Length field in the buggy
origin server response and serve a hit with other updated headers.

This is a known problem but nobody has volunteered to address it yet.

For workarounds, see https://bugs.squid-cache.org/show_bug.cgi?id=4882#c14

HTH,

Alex.

>> On 24 Feb 2021, at 13:50, Señor J Onion <senor.j.onion at gmail.com> wrote:
>>
>> I am new to Squid, I have been trying to get this to work for almost two weeks now, and have found nothing in the archives.
>>
>> This is my curl command (you will get a 403 forbidden by the time you run this dear reader):
>>
>> curl -s -D - -o /dev/null -G -d "alt=media" -x "http://localhost:3128" http://storage.googleapis.com/omgimg.appspot.com/test.jpeg -H "host:storage.googleapis.com" -H "x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" -H "x-amz-date:20210224T111631Z" -H "authorization:AWS4-HMAC-SHA256 Credential=GOOG1EGG4VCQ2EVRCJ2JCIO7ZDSZ3CY45Q72ATYZU2P32HITBFUOVQ6TEBWXI/20210224/auto/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=77a60480e47dda2b65ef3ebcd72a032458685e74e2560bb9083dbb03c3f6c13d”
>>
>>
>> These are the HTTP response headers:
>>
>>
>> FIRST RUN:
>>
>> HTTP/1.1 200 OK
>> X-GUploader-UploadID: ABg5-UwzmWjdWRPkonOxhLl3rUUik6wN3MB_ME1w1pxS5Rtmp_Cl1AAiP5G3tA9oXpFfAMnLCn5Pb8VW1mioc6GI-wJDun1S_g
>> Expires: Wed, 24 Feb 2021 11:20:08 GMT
>> Date: Wed, 24 Feb 2021 11:20:08 GMT
>> Cache-Control: private, max-age=0
>> Last-Modified: Tue, 04 Aug 2020 12:09:00 GMT
>> ETag: "d5b65c332fb6f80a0eade692b40b4afd"
>> Content-Type: image/jpeg
>> x-goog-hash: crc32c=6ijxaQ==
>> x-goog-hash: md5=1bZcMy+2+AoOreaStAtK/Q==
>> x-goog-storage-class: STANDARD
>> Accept-Ranges: bytes
>> Content-Length: 2296040
>> Server: UploadServer
>> X-Cache: MISS from 80396e157a13
>> X-Cache-Lookup: MISS from 80396e157a13:3128
>> Via: 1.1 80396e157a13 (squid/3.5.27)
>> Connection: keep-alive
>>
>> Squid log: 200 2296721 TCP_MISS:HIER_DIRECT
>>
>>
>> SECOND RUN:
>>
>> HTTP/1.1 200 OK
>> Content-Type: image/jpeg
>> x-goog-hash: crc32c=6ijxaQ==
>> x-goog-hash: md5=1bZcMy+2+AoOreaStAtK/Q==
>> x-goog-storage-class: STANDARD
>> Accept-Ranges: bytes
>> X-GUploader-UploadID: ABg5-UzooDZGnVTXxGIWQ2i25EasnR3glFz41FfUFvclACmZb3iDccpSXsGbRH0cr-8lofOc6Wb3knUzYMTgj_wdrzo
>> Expires: Wed, 24 Feb 2021 11:20:17 GMT
>> Date: Wed, 24 Feb 2021 11:20:17 GMT
>> Cache-Control: private, max-age=0
>> Last-Modified: Tue, 04 Aug 2020 12:09:00 GMT
>> ETag: "d5b65c332fb6f80a0eade692b40b4afd"
>> Content-Length: 0
>> Server: UploadServer
>> Age: 0
>> X-Cache: HIT from 80396e157a13
>> X-Cache-Lookup: HIT from 80396e157a13:3128
>> Via: 1.1 80396e157a13 (squid/3.5.27)
>> Connection: keep-alive
>>
>> Squid log: 200 651397 TCP_REFRESH_UNMODIFIED_ABORTED:HIER_DIRECT
>>
>>
>> I don’t know why the second time round, I a) don’t get a HIT, and b) why my content-length is 0.
>>
>> I think - this is the reason why - when I make these same calls in nodejs I end up with an HTTP Parse error (HPE_INVALID_CONSTANT).
>>
>>
>> My squid.conf file looks like this:
>>
>> acl localnet src 0.0.0.1-0.255.255.255
>> acl localnet src 10.0.0.0/8
>> acl localnet src 100.64.0.0/10
>> acl localnet src 169.254.0.0/16
>> acl localnet src 172.16.0.0/12
>> acl localnet src 192.168.0.0/16
>> acl localnet src fc00::/7
>> acl localnet src fe80::/10
>>
>> acl SSL_ports port 443
>> acl Safe_ports port 80
>> acl Safe_ports port 443
>>
>> acl Connect_ports port 80
>> acl Connect_ports port 443
>>
>> acl CONNECT method CONNECT
>>
>> http_access deny !Safe_ports
>>
>> http_access deny CONNECT !Connect_ports
>>
>> http_access allow localhost manager
>> http_access deny manager
>>
>> http_access deny to_localhost
>>
>> http_access allow localnet
>> http_access allow localhost
>>
>> http_access deny all
>>
>> strip_query_terms off
>> refresh_pattern . 525600 100% 525600 override-expire override-lastmod ignore-reload ignore-no-cache ignore-no-store reload-into-ims ignore-must-revalidate ignore-private ignore-auth store-stale
>>
>> cache_mem 2500 MB
>> maximum_object_size_in_memory 100 MB
>> memory_cache_mode always
>>
>>
>>
>>
>> Any direction would be greatly appreciated!
>>
>>
>>
>>
> 
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> 