[squid-users] Squid Memory Leak with certain FTP requests?

[Yuri Voinov]

Squid first saves object in memory. Then swapout object to cache. As usual:

# MEMORY CACHE OPTIONS
# 
-----------------------------------------------------------------------------

#  TAG: cache_mem    (bytes)
#    NOTE: THIS PARAMETER DOES NOT SPECIFY THE MAXIMUM PROCESS SIZE.
#    IT ONLY PLACES A LIMIT ON HOW MUCH ADDITIONAL MEMORY SQUID WILL
#    USE AS A MEMORY CACHE OF OBJECTS. SQUID USES MEMORY FOR OTHER
#    THINGS AS WELL. SEE THE SQUID FAQ SECTION 8 FOR DETAILS.
#
#    'cache_mem' specifies the ideal amount of memory to be used
#    for:
#        * In-Transit objects
#        * Hot Objects
#        * Negative-Cached objects
#
#    Data for these objects are stored in 4 KB blocks.  This
#    parameter specifies the ideal upper limit on the total size of
#    4 KB blocks allocated.  In-Transit objects take the highest
#    priority.
#
#    In-transit objects have priority over the others.  When
#    additional space is needed for incoming data, negative-cached
#    and hot objects will be released.  In other words, the
#    negative-cached and hot objects will fill up any unused space
#    not needed for in-transit objects.
#
#    If circumstances require, this limit will be exceeded.
#    Specifically, if your incoming request rate requires more than
#    'cache_mem' of memory to hold in-transit objects, Squid will
#    exceed this limit to satisfy the new requests.  When the load
#    decreases, blocks will be freed until the high-water mark is
#    reached.  Thereafter, blocks will be used to store hot
#    objects.
#
#    If shared memory caching is enabled, Squid does not use the shared
#    cache space for in-transit objects, but they still consume as much
#    local memory as they need. For more details about the shared memory
#    cache, see memory_cache_shared.
#Default:
# cache_mem 256 MB

#  TAG: maximum_object_size_in_memory    (bytes)
#    Objects greater than this size will not be attempted to kept in
#    the memory cache. This should be set high enough to keep objects
#    accessed frequently in memory to improve performance whilst low
#    enough to keep larger objects from hoarding cache_mem.
#Default:
# maximum_object_size_in_memory 512 KB

#  TAG: memory_cache_shared    on|off
#    Controls whether the memory cache is shared among SMP workers.
#
#    The shared memory cache is meant to occupy cache_mem bytes and replace
#    the non-shared memory cache, although some entities may still be
#    cached locally by workers for now (e.g., internal and in-transit
#    objects may be served from a local memory cache even if shared memory
#    caching is enabled).
#
#    By default, the memory cache is shared if and only if all of the
#    following conditions are satisfied: Squid runs in SMP mode with
#    multiple workers, cache_mem is positive, and Squid environment
#    supports required IPC primitives (e.g., POSIX shared memory segments
#    and GCC-style atomic operations).
#
#    To avoid blocking locks, shared memory uses opportunistic algorithms
#    that do not guarantee that every cachable entity that could have been
#    shared among SMP workers will actually be shared.
#
#    Currently, entities exceeding 32KB in size cannot be shared.
#Default:
# "on" where supported if doing memory caching with multiple SMP workers.

#  TAG: memory_cache_mode
#    Controls which objects to keep in the memory cache (cache_mem)
#
#    always    Keep most recently fetched objects in memory (default)
#
#    disk    Only disk cache hits are kept in memory, which means
#        an object must first be cached on disk and then hit
#        a second time before cached in memory.
#
#    network    Only objects fetched from network is kept in memory
#Default:
# Keep the most recently fetched objects in memory

This is no memory leaking, but normal cache behaviour. As documented.

You can play around with range_offset_limit and quick_abort_min parameters.

Or try to no cache this FTP with ACL.

Usually, when suggests memory leaking, this often OS issue. Not Squid.

11.02.15 16:02, Silamael пишет:
> Hi,
>
> One of our customers does constantly mirroring of some FTP directories
> and noticed a huge memory consumption of Squid. As far as I can see with
> squidclient mgr:mem, the 2K buffers are constantly increasing if Squid
> is processing FTP requests like wget ftp://some.server/pub/ or wget -m
> ftp://...
> In my tests the first variant resulted in an increase of 29 kb per request.
> If I directly request a certain file, the increase of the 2K buffers
> seems not to happen.
> I have reproduced this behaviour with both squid 3.2.13 and also 3.4.6
> and 3.4.11.
>
> Greeting,
> Matthias
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users