[squid-users] squid logging disable based on ACL & kernel: Out of memory

Wed May 6 12:58:17 UTC 2020

Hi Alex,

I updated to latest squid as you suggested, and I tried SSL-Bump using
below config (which filters URLs which are in 443 too), however I have 600
users (windows, linux, Mac, mobile OS like Androd, Windows etc), so asking
them to import CA certificate in browser is not feasible.

1. Is there any way to filter HTTPS URLs without importing CA certificates
on client side? if available can you share config snippet
2. for 16GB RAM, 4 core CPU, 8GB Swap, expected to have 10GB cache,  how to
calculate configurations parameters, is there any thumb rule ? please share
how you usually calculate.

# config
cache_mgr webmaster
cache deny QUERY
cache_mem 256 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size 4 MB
minimum_object_size 0 KB
maximum_object_size_in_memory 512 kB
ipcache_size 2048
ipcache_low 90
ipcache_high 95
fqdncache_size 1024
cache_replacement_policy lru
memory_replacement_policy lru
cache_dir ufs /var/spool/squid 10000 16 256
cache_effective_user squid
cache_effective_group squid
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
memory_pools on
memory_pools_limit 5 MB

# SSL-Bump -working but not feasible.
http_port 3128 ssl-bump cert=/etc/squid/sslcert/proxyCA.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/lib64/squid/security_file_certgen -s
 /var/spool/squid/ssl_db -M 4MB
sslcrtd_children 5
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all

------------------------------------ My New Environment --------------------
# squid -v
Squid Cache: Version 4.4
Service Name: squid

# cat /etc/redhat-release
CentOS Linux release 8.1.1911 (Core)

# Tested ACLs
logformat test_log %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %>ru %[un %Sh/%<a
%mt
acl test_sites dstdomain "/etc/squid/acls/test_sites.acl"
access_log /var/log/squid/test_site.log test_log test_sites

# tail -f /var/log/squid/test_site.log
1588678050.178   3247 10.0.2.15 TCP_TUNNEL/200 28073 CONNECT
nav.sciencedirect.com:443 akshay HIER_DIRECT/91.235.133.74 -
1588678050.189   3942 10.0.2.15 TCP_TUNNEL/200 24000 CONNECT
nav.sciencedirect.com:443 akshay HIER_DIRECT/91.235.133.74 -
1588678050.355   2552 10.0.2.15 TCP_TUNNEL/200 788 CONNECT
nav.sciencedirect.com:443 akshay HIER_DIRECT/91.235.133.74 -
1588681419.635    647 10.0.2.15 TCP_MISS/200 402 POST
http://scratchpads.eu/modules/statistics/statistics.php akshay HIER_DIRECT/
157.140.2.32 text/html
1588681420.055   1069 10.0.2.15 TCP_MISS/200 46772 GET
http://scratchpads.eu/sites/all/themes/scratchpads_eu/images/shrimp-202px.png
akshay HIER_DIRECT/157.140.2.32 image/png

On Sat, May 2, 2020 at 1:00 AM Alex Rousskov <
rousskov at measurement-factory.com> wrote:

> On 5/1/20 12:43 PM, Akshay Hegde wrote:
>
> > I have below option globally, which I don't want to make "off"
> > strip_query_terms on
>
> > acl track dstdomain "/etc/squid/sites_track.txt"
> > access_log /var/log/squid/full_site_links.log squid_custom track
>
> > however for specific ACL I would like to log full URL with query
> > parameters, how this can be done ?
>
> I have not tested this, and the results may be version-dependent, but
> according to logformat documentation[1], %ru honors strip_query_terms
> while %>ru does not:
>
>     logformat strippedFormat %ts... %ru ...
>     access_log ... strippedFormat track !specific_ACL
>
>     logformat detailedFormat %ts... %>ru ...
>     access_log ... detailedFormat track specific_ACL
>
> [1] http://www.squid-cache.org/Doc/config/logformat/
>
>
> HTH,
>
> Alex.
>
> > On Fri, May 1, 2020 at 7:05 PM Alex Rousskov wrote:
> >
> >     On 5/1/20 1:20 AM, Akshay Hegde wrote:
> >
> >     > *1. How to disable logging of few ACLs ?
> >
> >     Use "access_log none aclX" to prevent creation of access.log records
> for
> >     transactions matching aclX. See
> >
> http://lists.squid-cache.org/pipermail/squid-users/2020-April/021876.html
> >     for
> >     some related caveats.
> >
> >
> >     > *2. Kernel Out of Memory
> >
> >     This problem is most likely unrelated to logging. If your Squid is
> >     gradually leaking memory (rather than just being overwhelmed with
> >     traffic), then the first step towards removing those memory leaks
> would
> >     be to upgrade your Squid from the unsupported and buggy v3.1.10.
> >
> >
> >     HTH,
> >
> >     Alex.
> >
> >
> >
> > --
> > <
> https://about.me/akshay.k.hegde?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=edit_panel&utm_content=thumb
> >
> >
> > Akshay Hegde
> > about.me/akshay.k.hegde
> > <
> https://about.me/akshay.k.hegde?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=edit_panel&utm_content=thumb
> >
> >
> >
>
>

-- 
<https://about.me/akshay.k.hegde?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=edit_panel&utm_content=thumb>
Akshay Hegde
about.me/akshay.k.hegde
<https://about.me/akshay.k.hegde?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=edit_panel&utm_content=thumb>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20200506/b710bb17/attachment.html>