[squid-users] cacheNumbObject unreasonably small with rock cache

[Alex Rousskov]

On 2025-04-02 13:05, Dave Dykstra wrote:
> Yes that helps a lot, Alex.  I do see at the end of mgr:info:
> 
> Internal Data Structures:
> 	   576 StoreEntries
> 	   576 StoreEntries with MemObjects
> 	557676 Hot Object Cache Items
> 	584657 on-disk objects
> 
> We'll work on a pull request to change that to on-disk objects for rock cache.

Strike the last three words: As I strongly implied, current SNMP 
cacheNumObjCount stats are wrong for configurations using UFS-based 
caches, configurations using rock caches, and non-caching configurations.


Cheers,

Alex.


> On Wed, Apr 02, 2025 at 11:21:15AM -0400, Alex Rousskov wrote:
>> On 2025-04-02 10:45, Dave Dykstra wrote:
>>
>>> We're trying rock cache for the first time, on squid 6.13.  The
>>> machine is quite large and heavily used, with 10 workers configured,
>>> 140G of shared memory cache, and 500G of rock cache configured.
>>> However, the cacheNumObject SNMP counter is staying quite steadily at
>>> around only 570 even as the cache grows.  Currently du shows the rock
>>> file taking 40G, and the mgr:storedir statistics show 448k entries.
>>
>> I assume you are asking about an SNMP measurement that Squid MIB file
>> calls cacheNumObjCount (node 1.3.6.1.4.1.3495.1.3.1).
>>
>> Today, that measurement reports the number of StoreEntry objects
>> currently in use. You should see similar stats on the "StoreEntry" line
>> in "Internal Data Structures" section of mgr:info cache manager report.
>>
>> Roughly speaking:
>>
>> * For UFS-based caches and non-SMP memory caches, that number is often
>> (but not always!) close to the number of cached objects because these
>> caches create a StoreEntry object for every object stored in the cache.
>> That is how these stores index cached content.
>>
>> * For rock caches and SMP memory caches, that number is usually very
>> different from the number of cached objects because these caches do not
>> create StoreEntry objects when indexing cached content.
>>
>> * StoreEntry objects are not limited to caches. Many transactions,
>> including many transactions that do not retrieve cache hits and do not
>> store cache misses create StoreEntry objects. In related Squid
>> terminology, "Store" is not just "cache", but a more general medium for
>> passing HTTP responses around.
>>
>> I have not checked all the necessary details, but I suspect that SNMP
>> code needs to be fixed to report the same stats that are currently
>> available on "on-disk objects" line in "Internal Data Structures"
>> section of mgr:info cache manager report.
>>
>> I hope that mgr:storedir statistics that you have mentioned also
>> reflects the actual disk cache usage, at least for rock caches.
>>
>>
>> HTH,
>>
>> Alex.
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> https://lists.squid-cache.org/listinfo/squid-users
>>