[squid-users] squid logging disable based on ACL & kernel: Out of memory

Wed May 6 14:45:55 UTC 2020

Hi Alex,

Thanks for confirming, I lost hope. Can you share some link or details
about below

> There are other ways to police
traffic (e.g., browser plugins), but they all require fiddling with the
client environment.

On Wed, May 6, 2020, 7:56 PM Alex Rousskov <rousskov at measurement-factory.com>
wrote:

> On 5/6/20 8:58 AM, Akshay Hegde wrote:
>
> > 1. Is there any way to filter HTTPS URLs without importing CA
> > certificates on client side?
>
> No, there is no way for a proxy to look at request URLs without the
> browser trusting the proxy certificate. There are other ways to police
> traffic (e.g., browser plugins), but they all require fiddling with the
> client environment.
>
>
> > 2. for 16GB RAM, 4 core CPU, 8GB Swap, expected to have 10GB cache,  how
> > to calculate configurations parameters, is there any thumb rule ?
>
> I believe there is some related advice on Squid wiki:
> https://wiki.squid-cache.org/SquidFaq/SquidMemory
>
> HTH,
>
> Alex.
>
>
> > # config
> > cache_mgr webmaster
> > cache deny QUERY
> > cache_mem 256 MB
> > cache_swap_low 90
> > cache_swap_high 95
> > maximum_object_size 4 MB
> > minimum_object_size 0 KB
> > maximum_object_size_in_memory 512 kB
> > ipcache_size 2048
> > ipcache_low 90
> > ipcache_high 95
> > fqdncache_size 1024
> > cache_replacement_policy lru
> > memory_replacement_policy lru
> > cache_dir ufs /var/spool/squid 10000 16 256
> > cache_effective_user squid
> > cache_effective_group squid
> > cache_log /var/log/squid/cache.log
> > cache_store_log /var/log/squid/store.log
> > memory_pools on
> > memory_pools_limit 5 MB
> >
> > # SSL-Bump -working but not feasible.
> > http_port 3128 ssl-bump cert=/etc/squid/sslcert/proxyCA.pem
> > generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> > sslcrtd_program /usr/lib64/squid/security_file_certgen -s
> >  /var/spool/squid/ssl_db -M 4MB
> > sslcrtd_children 5
> > acl step1 at_step SslBump1
> > ssl_bump peek step1
> > ssl_bump bump all
> >
> > ------------------------------------ My New Environment
> --------------------
> > # squid -v
> > Squid Cache: Version 4.4
> > Service Name: squid
> >
> > # cat /etc/redhat-release
> > CentOS Linux release 8.1.1911 (Core)
> >
> >
> > # Tested ACLs
> > logformat test_log %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %>ru %[un
> > %Sh/%<a %mt
> > acl test_sites dstdomain "/etc/squid/acls/test_sites.acl"
> > access_log /var/log/squid/test_site.log test_log test_sites
> >
> > # tail -f /var/log/squid/test_site.log
> > 1588678050.178   3247 10.0.2.15 TCP_TUNNEL/200 28073 CONNECT
> > nav.sciencedirect.com:443 <http://nav.sciencedirect.com:443> akshay
> > HIER_DIRECT/91.235.133.74 <http://91.235.133.74> -
> > 1588678050.189   3942 10.0.2.15 TCP_TUNNEL/200 24000 CONNECT
> > nav.sciencedirect.com:443 <http://nav.sciencedirect.com:443> akshay
> > HIER_DIRECT/91.235.133.74 <http://91.235.133.74> -
> > 1588678050.355   2552 10.0.2.15 TCP_TUNNEL/200 788 CONNECT
> > nav.sciencedirect.com:443 <http://nav.sciencedirect.com:443> akshay
> > HIER_DIRECT/91.235.133.74 <http://91.235.133.74> -
> > 1588681419.635    647 10.0.2.15 TCP_MISS/200 402 POST
> > http://scratchpads.eu/modules/statistics/statistics.php akshay
> > HIER_DIRECT/157.140.2.32 <http://157.140.2.32> text/html
> > 1588681420.055   1069 10.0.2.15 TCP_MISS/200 46772 GET
> >
> http://scratchpads.eu/sites/all/themes/scratchpads_eu/images/shrimp-202px.png
> > akshay HIER_DIRECT/157.140.2.32 <http://157.140.2.32> image/png
> >
> >
> >
> >
> > On Sat, May 2, 2020 at 1:00 AM Alex Rousskov
> > <rousskov at measurement-factory.com
> > <mailto:rousskov at measurement-factory.com>> wrote:
> >
> >     On 5/1/20 12:43 PM, Akshay Hegde wrote:
> >
> >     > I have below option globally, which I don't want to make "off"
> >     > strip_query_terms on
> >
> >     > acl track dstdomain "/etc/squid/sites_track.txt"
> >     > access_log /var/log/squid/full_site_links.log squid_custom track
> >
> >     > however for specific ACL I would like to log full URL with query
> >     > parameters, how this can be done ?
> >
> >     I have not tested this, and the results may be version-dependent, but
> >     according to logformat documentation[1], %ru honors strip_query_terms
> >     while %>ru does not:
> >
> >         logformat strippedFormat %ts... %ru ...
> >         access_log ... strippedFormat track !specific_ACL
> >
> >         logformat detailedFormat %ts... %>ru ...
> >         access_log ... detailedFormat track specific_ACL
> >
> >     [1] http://www.squid-cache.org/Doc/config/logformat/
> >
> >
> >     HTH,
> >
> >     Alex.
> >
> >     > On Fri, May 1, 2020 at 7:05 PM Alex Rousskov wrote:
> >     >
> >     >     On 5/1/20 1:20 AM, Akshay Hegde wrote:
> >     >
> >     >     > *1. How to disable logging of few ACLs ?
> >     >
> >     >     Use "access_log none aclX" to prevent creation of access.log
> >     records for
> >     >     transactions matching aclX. See
> >     >
> >
> http://lists.squid-cache.org/pipermail/squid-users/2020-April/021876.html
> >     >     for
> >     >     some related caveats.
> >     >
> >     >
> >     >     > *2. Kernel Out of Memory
> >     >
> >     >     This problem is most likely unrelated to logging. If your
> Squid is
> >     >     gradually leaking memory (rather than just being overwhelmed
> with
> >     >     traffic), then the first step towards removing those memory
> >     leaks would
> >     >     be to upgrade your Squid from the unsupported and buggy
> v3.1.10.
> >     >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20200506/6151953a/attachment.html>