[squid-users] Squid Transparent/intercept Issues
christian brendan
bosscb.chrisbren at gmail.com
Tue Mar 21 11:00:05 UTC 2017
Re: Squid Transparent/intercept Issues
On Tue, Mar 21, 2017 at 8:05 AM, <squid-users-request at lists.squid-cache.org>
wrote:
> Send squid-users mailing list submissions to
> squid-users at lists.squid-cache.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.squid-cache.org/listinfo/squid-users
> or, via email, send a message with subject or body 'help' to
> squid-users-request at lists.squid-cache.org
>
> You can reach the person managing the list at
> squid-users-owner at lists.squid-cache.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of squid-users digest..."
>
>
> Today's Topics:
>
> 1. Re: Squid Transparent/intercept Issues (Antony Stone)
> 2. Re: SMP and AUFS (Matus UHLAR - fantomas)
> 3. Re: SMP and AUFS (Alex Rousskov)
> 4. Re: squid workers question (Alex Rousskov)
> 5. Re: squid workers question (Matus UHLAR - fantomas)
> 6. Re: SSL Bump issues (Alex Rousskov)
> 7. blocking or allowing specific youtube videos (Sohan Wijetunga)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 20 Mar 2017 16:56:17 +0100
> From: Antony Stone <Antony.Stone at squid.open.source.it>
> To: squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] Squid Transparent/intercept Issues
> Message-ID: <201703201656.18291.Antony.Stone at squid.open.source.it>
> Content-Type: Text/Plain; charset="iso-8859-15"
>
> On Monday 20 March 2017 at 16:26:40, christian brendan wrote:
>
> > Hello Everyone,
> >
> > Squid Cache: Version 3.5.20
> > OS: CentOS 7
> >
> > I have used squid for quite some times non transparently and it works,
> > problem kicks in when: http_port 3128 transparent is enabled.
> > Access denied error page shows up when transparent is enabled
> > ERRORThe requested URL could not be retrieved
>
> How are you getting the packets to the Squid server for interception?
>
> Is the Squid server in the default route between your clients and the
> Internet, or are you redirecting the packets to the Squid server somehow?
>
> Please give *details* of how you are intercepting and sending the packets
> to
> Squid (eg: iptables rules, and which machine/s the rules are running on).
>
>
> Antony.
>
> --
> Anything that improbable is effectively impossible.
>
> - Murray Gell-Mann, Nobel Prizewinner in Physics
>
> Please reply to the
> list;
> please *don't* CC
> me.
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 20 Mar 2017 17:15:16 +0100
> From: Matus UHLAR - fantomas <uhlar at fantomas.sk>
> To: squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] SMP and AUFS
> Message-ID: <20170320161516.GB26154 at fantomas.sk>
> Content-Type: text/plain; charset=us-ascii; format=flowed
>
> On 19.03.17 11:08, Alex Rousskov wrote:
> >On 03/18/2017 11:11 PM, senor wrote:
> >
> >> There are many references in the squid wiki, FAQ and Knowlegebase about
> >> SMP but I don't see any of them reflecting the concerns you have brought
> >> up.
> >
> >There is a paragraph about these problems at [1] (search for "ufs") but
> >I agree that better documentation, including wiki and
> >squid.conf.documented changes/additions would be nice.
> >
> > [1] http://wiki.squid-cache.org/Features/SmpScale
> >
> >
> >> My point in mentioning that there are a lot of installations using
> >> SMP and AUFS is that something widely used but buggy tends to be brought
> >> up on this email list and I haven't seen it.
> >
> >IIRC, it has been brought up several times on the mailing lists and in
> >Bugzilla. Once you dedicate each ufs-based store to each individual
> >worker, most of the problems become subtle, often "invisible" to an
> >admin because they "break" transactions, not Squid, especially if you do
> >not use a mixture of ufs-based and rock stores. Using mailing list as an
> >indicator that as subtle problem does _not_ exist is a risky strategy IMO.
>
> Well, I personally will still be curious how much does SMP affect the case
> of
> one worker and one or more diskers...
>
> do diskers only provide I/O to the requestor?
>
> --
> Matus UHLAR - fantomas, uhlar at fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Depression is merely anger without enthusiasm.
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 20 Mar 2017 12:19:58 -0600
> From: Alex Rousskov <rousskov at measurement-factory.com>
> To: squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] SMP and AUFS
> Message-ID:
> <cd47a96b-357d-8cfd-41e4-d4d376da10c1 at measurement-factory.com>
> Content-Type: text/plain; charset=utf-8
>
> On 03/20/2017 10:15 AM, Matus UHLAR - fantomas wrote:
>
> > Well, I personally will still be curious how much does SMP affect the
> > case of one worker and one or more diskers...
>
> I do not understand why you are asking this question in AUFS context.
> AUFS does not use diskers! Today, only Rock store uses diskers (in SMP
> mode). Some other [ufs-based] cache stores use various helper threads
> and processes for I/O as well, but those helper processes are not
> diskers or even kids in SMP terminology.
>
>
> > do diskers only provide I/O to the requestor?
>
> Diskers primary function is low-level disk cache I/O. Like all kids,
> diskers respond to cache manager requests and Squid management events
> (e.g. shutdown and reconfiguration). IIRC, diskers also build in-RAM
> cache_dir index.
>
> http://wiki.squid-cache.org/Features/SmpScale#Terminology
>
> HTH,
>
> Alex.
>
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 20 Mar 2017 12:32:44 -0600
> From: Alex Rousskov <rousskov at measurement-factory.com>
> To: squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] squid workers question
> Message-ID:
> <5c14decf-fd76-b6cb-a497-85b4e226b34c at measurement-factory.com>
> Content-Type: text/plain; charset=utf-8
>
> On 03/20/2017 09:20 AM, Matus UHLAR - fantomas wrote:
> > On 10.03.17 08:52, Alex Rousskov wrote:
> >> Sorry, but that 2010 documentation is outdated. It was written before
> >> Rock store, a 2011 feature that changed what "SMP mode" means. This is
> >> my fault. Here is a replacement draft that I was working on until wiki
> >> went down:
> >>
> >>> NAME: workers
> >>> DEFAULT: 1
> >>> Number of main Squid processes or "workers" to fork and maintain.
> >>>
> >>> In a typical setup, each worker listens on all http_port(s) and
> >>> proxies requests without talking to other workers. Depending on
> >>> configuration, other Squid processes (e.g., rock store "diskers")
> >>> may also participate in request processing. All such Squid
> processes
> >>> are collectively called "kids".
> >>>
> >>> Setting workers to 0 disables kids creation and is similar to
> >>> running "squid -N ...". A positive value starts that many workers.
>
> > The default of 1 (only) creates kids for each rock store configured.
>
> What makes you think that? I believe "workers 1" in the presence of rock
> cache_dirs should create one kid to handle HTTP transaction _plus_ one
> kid for each rock cache_dir.
>
>
> >>> When multiple concurrent kids are in use, Squid is said to work in
> >>> "SMP mode". Some Squid features (e.g., ufs-based cache_dirs) are
> not
> >>> SMP-aware and should not or cannot be used in SMP mode.
> >>>
> >>> See http://wiki.squid-cache.org/Features/SmpScale for details.
>
> > very nice, thanks. However this is not meant for the wiki, but for:
> > http://www.squid-cache.org/Doc/config/workers/
>
> To be more precise, the text is meant for src/cf.data.pre, from which
> squid.conf.documented (and Doc/Config pages) are generated from. Not
> sure why you say "However" though.
>
>
> > maybe that pages could be updated (all but 3.2 versions are the same).
>
> Once the above worker documentation changes are polished and committed
> to the Squid repository, the affected generated pages/files will be
> updated automatically.
>
> The documentation for earlier versions may never be updated though -- it
> depends on whether the changes are going to be ported and committed to
> the code branches corresponding to those earlier versions.
>
>
> >> The final version will probably move and extend the terminology-related
> >> text to the SMP section preamble -- it is kind of wrong to talk about
> >> diskers when documenting workers. Improvements and constructive
> >> suggestions welcomed!
> >
> > compared to current version I'd change it to:
> >
> > 1: start one main Squid process daemon (default)
> > "no SMP" when rock store is not used
> > "SMP" when rock store in use
>
> I agree that we should add something like this as a common-case example
> of general rules. Thank you.
>
> Alex.
>
>
>
> ------------------------------
>
> Message: 5
> Date: Mon, 20 Mar 2017 20:49:06 +0100
> From: Matus UHLAR - fantomas <uhlar at fantomas.sk>
> To: squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] squid workers question
> Message-ID: <20170320194906.GA30456 at fantomas.sk>
> Content-Type: text/plain; charset=us-ascii; format=flowed
>
> >> On 10.03.17 08:52, Alex Rousskov wrote:
> >>> Sorry, but that 2010 documentation is outdated. It was written before
> >>> Rock store, a 2011 feature that changed what "SMP mode" means. This is
> >>> my fault. Here is a replacement draft that I was working on until wiki
> >>> went down:
> >>>
> >>>> NAME: workers
> >>>> DEFAULT: 1
> >>>> Number of main Squid processes or "workers" to fork and maintain.
> >>>>
> >>>> In a typical setup, each worker listens on all http_port(s) and
> >>>> proxies requests without talking to other workers. Depending on
> >>>> configuration, other Squid processes (e.g., rock store "diskers")
> >>>> may also participate in request processing. All such Squid
> processes
> >>>> are collectively called "kids".
> >>>>
> >>>> Setting workers to 0 disables kids creation and is similar to
> >>>> running "squid -N ...". A positive value starts that many workers.
>
> >On 03/20/2017 09:20 AM, Matus UHLAR - fantomas wrote:
> >> The default of 1 (only) creates kids for each rock store configured.
>
> On 20.03.17 12:32, Alex Rousskov wrote:
> >What makes you think that? I believe "workers 1" in the presence of rock
> >cache_dirs should create one kid to handle HTTP transaction _plus_ one
> >kid for each rock cache_dir.
>
> That's exactly what I meant, for inclusion to your paragraph.
> Should I replace "kids" with "one extra kid"?
> and should I replace (only) by "however"?
>
> >>>> When multiple concurrent kids are in use, Squid is said to work in
> >>>> "SMP mode". Some Squid features (e.g., ufs-based cache_dirs) are
> not
> >>>> SMP-aware and should not or cannot be used in SMP mode.
> >>>>
> >>>> See http://wiki.squid-cache.org/Features/SmpScale for details.
> >
> >> very nice, thanks. However this is not meant for the wiki, but for:
> >> http://www.squid-cache.org/Doc/config/workers/
> >
> >To be more precise, the text is meant for src/cf.data.pre, from which
> >squid.conf.documented (and Doc/Config pages) are generated from. Not
> >sure why you say "However" though.
>
> You mentioned you were working on the draft until wiki went down.
> I understood the paragraph as replacement for "workers" documentation, not
> as something to be written to wiki...
>
> >> maybe that pages could be updated (all but 3.2 versions are the same).
> >
> >Once the above worker documentation changes are polished and committed
> >to the Squid repository, the affected generated pages/files will be
> >updated automatically.
> >
> >The documentation for earlier versions may never be updated though -- it
> >depends on whether the changes are going to be ported and committed to
> >the code branches corresponding to those earlier versions.
>
> it's up to the release team.
> I would recommend update the docs on the web to avoid issues for people
> using older squid versions, e.g. in enterprise environment
>
> >>> The final version will probably move and extend the terminology-related
> >>> text to the SMP section preamble -- it is kind of wrong to talk about
> >>> diskers when documenting workers. Improvements and constructive
> >>> suggestions welcomed!
> >>
> >> compared to current version I'd change it to:
> >>
> >> 1: start one main Squid process daemon (default)
> >> "no SMP" when rock store is not used
> >> "SMP" when rock store in use
> >
> >I agree that we should add something like this as a common-case example
> >of general rules. Thank you.
>
> if we replace the current paragraph with your proposed one, I have proposed
> change at the top
>
> --
> Matus UHLAR - fantomas, uhlar at fantomas.sk ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Eagles may soar, but weasels don't get sucked into jet engines.
>
>
> ------------------------------
>
> Message: 6
> Date: Mon, 20 Mar 2017 14:08:48 -0600
> From: Alex Rousskov <rousskov at measurement-factory.com>
> To: squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] SSL Bump issues
> Message-ID:
> <d729abc8-9a3a-25e0-9185-d1cdbd2d91cc at measurement-factory.com>
> Content-Type: text/plain; charset=utf-8
>
> On 03/19/2017 07:58 PM, mr_jrt wrote:
>
> > ...but the only way I've got any successful SSL proxying is with:
> >
> >
> > ...but as expected, that's clearly not doing any bumping from the logs:
> >
> >
> >
> > When I put anything more in, i.e.
> >
> >
> > Then it turns on the mode:
> >
> >
> > ...but then I just get errors about no ciphers:
> >
>
> Please note that your configuration and other details in the post did
> not get through to the mailing list (probably due to some fancy quoting
> provided by Nabble that does not get through to the actual squid-users
> mailing list).
>
> Alex.
>
>
>
> ------------------------------
>
> Message: 7
> Date: Tue, 21 Mar 2017 12:35:25 +0530
> From: Sohan Wijetunga <sohanwijetunga at gmail.com>
> To: squid-users at lists.squid-cache.org
> Subject: [squid-users] blocking or allowing specific youtube videos
> Message-ID:
> <CAOUuUH671PqQQF4sd9ykGarqFiVOp_TZ8HMs6GfEBh3QTVjkwA at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Project subject is blocking or allowing specific youtube videos. For that
> research I hope to add more features but currently I’m stuck to take full
> urls from clients. According to my project, environment should be client
> server environment. All the client’s youtube traffic should be manage
> through the gateway. I currently following squid helper programs it seems
> to be fulfil my requirement but those examples are not enough for testing.
> Using of squid helper program is to do some development in my research
> future. I really need to do that project using squid.
>
>
>
> I look forward to hearing from you soon.
>
> Thank you.
>
> Best Regards,
>
> Sohan.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.squid-cache.org/pipermail/squid-users/
> attachments/20170321/435d3a19/attachment.html>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
>
> ------------------------------
>
> End of squid-users Digest, Vol 31, Issue 59
> *******************************************
>
@Antony.Stone
1. I am using mikrotik routerboard to redirect traffic, with this rule:
dd action=dst-nat chain=dstnat comment="Redirect port 80 to SquidProxy"
dst-port=80 protocol=tcp \ src-address=10.24.7.100 to-addresses=10.24.7.101
to-ports=3128
3. It is not in default route, packets is been redirected.
4. There is no iptable rules, firewall is disabled for this test.
Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20170321/485abf31/attachment-0001.html>
More information about the squid-users
mailing list