[squid-users] Enable caching
Alex Rousskov
rousskov at measurement-factory.com
Tue Apr 11 13:22:28 UTC 2023
On 4/11/23 07:05, Andrey K wrote:
> I have written a simple perl script that outputs the contents of the
> metadata of the rock database slots Maybe it will be useful for other
> squid users.
> *rock_cache_dump.pl
Thanks a lot for sharing this neat script! I am sure it will be handy,
at least as a starting point, in use cases like yours.
> And maybe you will add it to new squid releases.
I would not. I believe this script is best maintained outside of the
official source code tree until we have the resources to properly polish
and maintain it.
Thank you,
Alex.
> чт, 6 апр. 2023 г. в 20:35, Alex Rousskov:
>
> On 4/6/23 09:08, Andrey K wrote:
>
> > Could you tell me if there is a way to view the objects (URLs)
> and their
> > statuses stored in the rock file?
>
> There is no visualization software for rock db storage. One can
> obviously use xxd and similar generic tools to look at raw db bytes,
> even in a running Squid instance, but your needs are probably different.
>
>
> > I tried unsuccessfully to find this information using squidclient in
> > mgr:menu.
>
> Cache manager queries are currently ineffective for analyzing
> individual
> rock cache_dir objects because cache manager code relies on the legacy
> in-memory worker-specific store index while rock uses shared memory
> structures shared among workers.
>
>
> > You gave me a very useful link:
> > https://wiki.squid-cache.org/Features/LargeRockStore
> <https://wiki.squid-cache.org/Features/LargeRockStore>
> > Maybe there is a more detailed description of the internal rock data
> > structures?
>
> IIRC, the next level of detail is available in source code only. For
> starting points, consider src/fs/rock/RockDbCell.h and the end of
> Rock::SwapDir::create() that writes an all-zeroes db file header.
>
>
> HTH,
>
> Alex.
>
>
> > I could try to write a script that reads the necessary
> information from
> > the cache_dir file.
> >
> > Kind regards,
> > Ankor.
> >
> >
> > ср, 5 апр. 2023 г. в 16:27, Alex Rousskov
> > <rousskov at measurement-factory.com
> <mailto:rousskov at measurement-factory.com>
> > <mailto:rousskov at measurement-factory.com
> <mailto:rousskov at measurement-factory.com>>>:
> >
> > On 4/5/23 06:07, Andrey K wrote:
> >
> > > Previously, caching was disabled on our proxy servers. Now we
> > need to
> > > cache some content (files about 10 MB in size).
> > > So we changed the squid.conf:
> >
> > > cache_dir ufs /data/squid/cache 32000 16 256 max-size=12000000
> > >
> > > We have 24 workers on each proxy.
> >
> > UFS-based cache_dirs are not supported in multi-worker
> configurations
> > and, in most cases, should not be used in such
> configurations. The
> > combination will violate basic HTTP caching rules and may
> crash Squid
> > and/or corrupt responses.
> >
> >
> > > We saw that some requests were taken from the cache, and some
> > were not.
> > > The documentation says:
> > > "In SMP configurations, cache_dir must not precede the workers
> > option
> > > and should use configuration macros or conditionals to
> give each
> > worker
> > > interested in disk caching a dedicated cache directory."
> >
> > The official documentation quoted above is stale and very
> misleading in
> > modern Squids. Ignore it. I will try to find the time to post
> a PR to
> > fix this.
> >
> >
> > > So we switched to a rock cache_dir:
> > > cache_dir rock /data/squid/cache 32000 max-size=12000000
> > >
> > > Now everything seems to be working fine in the test
> environment,
> > but I
> > > found limitations on the RockStore
> > > (https://wiki.squid-cache.org/Features/RockStore
> <https://wiki.squid-cache.org/Features/RockStore>
> > <https://wiki.squid-cache.org/Features/RockStore
> <https://wiki.squid-cache.org/Features/RockStore>>:
> > > "Objects larger than 32,000 bytes cannot be cached when
> > cache_dirs are
> > > shared among workers."
> >
> > The Feature/RockStore page is stale and can easily mislead.
> In general,
> > Feature/Foo wiki pages are often development-focused and get
> stale with
> > time. They cannot be reliably used as a Squid feature
> documentation.
> >
> >
> > > Does this mean that RockStore is not suitable for caching
> large
> > files?
> >
> > No, it does not. Rock storage has evolved since that Feature
> page was
> > written. You can see the following wiki page discussing
> evolved rock
> > storage design, but that page probably has some stale info as
> well:
> > https://wiki.squid-cache.org/Features/LargeRockStore
> <https://wiki.squid-cache.org/Features/LargeRockStore>
> > <https://wiki.squid-cache.org/Features/LargeRockStore
> <https://wiki.squid-cache.org/Features/LargeRockStore>>
> >
> >
> > > Should I switch back to the UFS and configure 24 cache_dirs
> >
> > If everything is "working fine", then you should not.
> Otherwise, I
> > recommend discussing specific problems before switching to that
> > unsupported and dangerous hack.
> >
> >
> > HTH,
> >
> > Alex.
> >
> > _______________________________________________
> > squid-users mailing list
> > squid-users at lists.squid-cache.org
> <mailto:squid-users at lists.squid-cache.org>
> > <mailto:squid-users at lists.squid-cache.org
> <mailto:squid-users at lists.squid-cache.org>>
> > http://lists.squid-cache.org/listinfo/squid-users
> <http://lists.squid-cache.org/listinfo/squid-users>
> > <http://lists.squid-cache.org/listinfo/squid-users
> <http://lists.squid-cache.org/listinfo/squid-users>>
> >
>
More information about the squid-users
mailing list