[squid-dev] Squid 4.13: too much memory used for ACL url_regex when big list file used
squid3 at treenet.co.nz
Tue Aug 17 01:38:23 UTC 2021
On 17/08/21 5:45 am, Meridoff wrote:
> Hello, I have simplest squid config with such acl:
> acl a1 url_regex "/tmp/urls.txt"
> In /tmp/urls.txt there are about 220 000 URL regexps, most of them in
> such form (example):
> There are a lot of memory used by squid for such configuration: about
> 2GB. Without this command: about 30MB used.
Are those numbers without traffic going through?
ie. just Squid loading the configuration then does nothing.
> So aprxm. 10KB for 1 regexp. Is it normal? I think it is too big..
regex rules get aggregated into longer compound lines, then compiled
into a binary form for the regex library to handle. It is not easy to
tell how the compiled form will compare in size to the original strings.
> I think that such big consumption is because of regexps , for simple
> strings (for example, domains or IPs) consumption will be less.
Sure. "Simple strings" will be not much larger than the size of the file
input - though there is some extra memory used for indexing etc.
> How I can decrease memory consumption for such big regexp lists in ACLs
> ? May be some fix in squid or some fix in regexp lists ?
First thing to do is figure out where the memory is being consumed.
Will produce a spreadhsheet in TSV format of the memory allocations your
Squid has. Look through that for the big memory spenders, with an eye on
the regex one(s) you are suspicious of.
More information about the squid-dev