[squid-users] squid logging disable based on ACL & kernel: Out of memory
Alex Rousskov
rousskov at measurement-factory.com
Wed May 6 14:51:50 UTC 2020
On 5/6/20 10:45 AM, Akshay Hegde wrote:
> Can you share some link or details about below
Sorry, I cannot -- it has been many years since I worked on browser
plugins, and I have heard that there were significant changes in
APIs/rights since then. Perhaps others on the mailing list can help you.
If not, most of the related information should be publicly available.
Alex.
>> There are other ways to police
> traffic (e.g., browser plugins), but they all require fiddling with the
> client environment.
>
> On Wed, May 6, 2020, 7:56 PM Alex Rousskov wrote:
>
> On 5/6/20 8:58 AM, Akshay Hegde wrote:
>
> > 1. Is there any way to filter HTTPS URLs without importing CA
> > certificates on client side?
>
> No, there is no way for a proxy to look at request URLs without the
> browser trusting the proxy certificate. There are other ways to police
> traffic (e.g., browser plugins), but they all require fiddling with the
> client environment.
>
>
> > 2. for 16GB RAM, 4 core CPU, 8GB Swap, expected to have 10GB
> cache, how
> > to calculate configurations parameters, is there any thumb rule ?
>
> I believe there is some related advice on Squid wiki:
> https://wiki.squid-cache.org/SquidFaq/SquidMemory
>
> HTH,
>
> Alex.
>
>
> > # config
> > cache_mgr webmaster
> > cache deny QUERY
> > cache_mem 256 MB
> > cache_swap_low 90
> > cache_swap_high 95
> > maximum_object_size 4 MB
> > minimum_object_size 0 KB
> > maximum_object_size_in_memory 512 kB
> > ipcache_size 2048
> > ipcache_low 90
> > ipcache_high 95
> > fqdncache_size 1024
> > cache_replacement_policy lru
> > memory_replacement_policy lru
> > cache_dir ufs /var/spool/squid 10000 16 256
> > cache_effective_user squid
> > cache_effective_group squid
> > cache_log /var/log/squid/cache.log
> > cache_store_log /var/log/squid/store.log
> > memory_pools on
> > memory_pools_limit 5 MB
> >
> > # SSL-Bump -working but not feasible.
> > http_port 3128 ssl-bump cert=/etc/squid/sslcert/proxyCA.pem
> > generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> > sslcrtd_program /usr/lib64/squid/security_file_certgen -s
> > /var/spool/squid/ssl_db -M 4MB
> > sslcrtd_children 5
> > acl step1 at_step SslBump1
> > ssl_bump peek step1
> > ssl_bump bump all
> >
> > ------------------------------------ My New Environment
> --------------------
> > # squid -v
> > Squid Cache: Version 4.4
> > Service Name: squid
> >
> > # cat /etc/redhat-release
> > CentOS Linux release 8.1.1911 (Core)
> >
> >
> > # Tested ACLs
> > logformat test_log %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %>ru %[un
> > %Sh/%<a %mt
> > acl test_sites dstdomain "/etc/squid/acls/test_sites.acl"
> > access_log /var/log/squid/test_site.log test_log test_sites
> >
> > # tail -f /var/log/squid/test_site.log
> > 1588678050.178 3247 10.0.2.15 TCP_TUNNEL/200 28073 CONNECT
> > nav.sciencedirect.com:443 <http://nav.sciencedirect.com:443>
> <http://nav.sciencedirect.com:443> akshay
> > HIER_DIRECT/91.235.133.74 <http://91.235.133.74>
> <http://91.235.133.74> -
> > 1588678050.189 3942 10.0.2.15 TCP_TUNNEL/200 24000 CONNECT
> > nav.sciencedirect.com:443 <http://nav.sciencedirect.com:443>
> <http://nav.sciencedirect.com:443> akshay
> > HIER_DIRECT/91.235.133.74 <http://91.235.133.74>
> <http://91.235.133.74> -
> > 1588678050.355 2552 10.0.2.15 TCP_TUNNEL/200 788 CONNECT
> > nav.sciencedirect.com:443 <http://nav.sciencedirect.com:443>
> <http://nav.sciencedirect.com:443> akshay
> > HIER_DIRECT/91.235.133.74 <http://91.235.133.74>
> <http://91.235.133.74> -
> > 1588681419.635 647 10.0.2.15 TCP_MISS/200 402 POST
> > http://scratchpads.eu/modules/statistics/statistics.php akshay
> > HIER_DIRECT/157.140.2.32 <http://157.140.2.32>
> <http://157.140.2.32> text/html
> > 1588681420.055 1069 10.0.2.15 TCP_MISS/200 46772 GET
> >
> http://scratchpads.eu/sites/all/themes/scratchpads_eu/images/shrimp-202px.png
> > akshay HIER_DIRECT/157.140.2.32 <http://157.140.2.32>
> <http://157.140.2.32> image/png
> >
> >
> >
> >
> > On Sat, May 2, 2020 at 1:00 AM Alex Rousskov
> > <rousskov at measurement-factory.com
> <mailto:rousskov at measurement-factory.com>
> > <mailto:rousskov at measurement-factory.com
> <mailto:rousskov at measurement-factory.com>>> wrote:
> >
> > On 5/1/20 12:43 PM, Akshay Hegde wrote:
> >
> > > I have below option globally, which I don't want to make "off"
> > > strip_query_terms on
> >
> > > acl track dstdomain "/etc/squid/sites_track.txt"
> > > access_log /var/log/squid/full_site_links.log squid_custom track
> >
> > > however for specific ACL I would like to log full URL with query
> > > parameters, how this can be done ?
> >
> > I have not tested this, and the results may be
> version-dependent, but
> > according to logformat documentation[1], %ru honors
> strip_query_terms
> > while %>ru does not:
> >
> > logformat strippedFormat %ts... %ru ...
> > access_log ... strippedFormat track !specific_ACL
> >
> > logformat detailedFormat %ts... %>ru ...
> > access_log ... detailedFormat track specific_ACL
> >
> > [1] http://www.squid-cache.org/Doc/config/logformat/
> >
> >
> > HTH,
> >
> > Alex.
> >
> > > On Fri, May 1, 2020 at 7:05 PM Alex Rousskov wrote:
> > >
> > > On 5/1/20 1:20 AM, Akshay Hegde wrote:
> > >
> > > > *1. How to disable logging of few ACLs ?
> > >
> > > Use "access_log none aclX" to prevent creation of access.log
> > records for
> > > transactions matching aclX. See
> > >
> >
> http://lists.squid-cache.org/pipermail/squid-users/2020-April/021876.html
> > > for
> > > some related caveats.
> > >
> > >
> > > > *2. Kernel Out of Memory
> > >
> > > This problem is most likely unrelated to logging. If
> your Squid is
> > > gradually leaking memory (rather than just being
> overwhelmed with
> > > traffic), then the first step towards removing those memory
> > leaks would
> > > be to upgrade your Squid from the unsupported and buggy
> v3.1.10.
> > >
>
More information about the squid-users
mailing list