[squid-users] External nat'ed transparent proxy

Tue Oct 25 16:26:54 UTC 2016

Hey Henry,

It's not about RFC at all from my point of view.
It's very simple to setup the system in a way that will work as you want but with Let say Ubuntu 16.04 or Debian 8(latest).
These are very stable in my environment and if you need some help with the design I would be able to assist you with it.
I cannot find right now the whole setup specs but it's very simple to mark connections by the VLAN or the source network:
You will just need to change the next rules to be static and to not rely on NFQUEUE:
http://wiki.squid-cache.org/EliezerCroitoru/Drafts/MwanLB#iptables_rules_example

Then write a special routing table per vlan.
The reason to do so is since this is how it is suppose to be and not because of the RFC.
Your setup breaks good connections but maybe you are just not aware of it.
I believe that migrating from 2.X to a 3.5 can be completed smoothly enough in a way that nobody will fill it.

Elezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il

-----Original Message-----
From: Henry Paulissen [mailto:henry at nitronetworks.nl] 
Sent: Tuesday, October 25, 2016 18:45
To: Eliezer Croitoru <eliezer at ngtech.co.il>
Cc: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] External nat'ed transparent proxy

Hi Eliezer,

Thank you for looking into it.

I fully understand the setup is far from ideal and breaks RFC's (I was already aware of this).

The reason we would like to keep this in place is to keep support on the older applications who aren't proxy aware. While still being able to keep some control of standard outbound connections. We know, that a sufficient knowledgeable hacker would very easily bypass the proxy but it is not in place to stop them. The proxy is there to stop the automatic bots / script kiddies / etc. A first line of defense.

That being said,
As a fast solution I now installed a debian weezy machine (who comes standard with squid 2.7.x). That proxy will act as transparent one and have the acls pushed to it from the new squid 3.x ones (so we can have a mixed env). That way we can keep supporting the older stuff (and have logging on it, who still uses it to encourage them to swap to the "real"
proxy).

Regards,

--
Henry Paulissen - PD0OM
mailto:henry at nitronetworks.nl - Phone: +31-(0)6-115.305.64 Linux/Unix System Engineer

On 25-10-16 16:38, Eliezer Croitoru wrote:
> Hey,
> 
> As Amos suggested you should use Policy based routing and not DNAT.
> The main reason for that is since it's breaking the Interception layer which squid relies on for fallback scenarios.
> 
> I can write the logic for this pretty fast but you should first understand that your setup is wrong in a way.
> 
> Eliezer
> 
> ----
> Eliezer Croitoru
> Linux System Administrator
> Mobile: +972-5-28704261
> Email: mailto:eliezer at ngtech.co.il
> 
> 
> -----Original Message-----
> From: Henry Paulissen [mailto:henry at nitronetworks.nl]
> Sent: Friday, September 30, 2016 12:48
> To: Eliezer Croitoru <mailto:eliezer at ngtech.co.il>
> Cc: mailto:squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] External nat'ed transparent proxy
> 
> Good morning Eliezer,
> 
> 
> It took some time for me to construct a drawing who would be 
> understandable enough how our setup is, as the diagrams you provided 
> didn't fully fit the case. But, I think I managed to make a 
> understandable drawing of it :-)
> 
> [ Link to PNG image ]
> https://drive.google.com/file/d/0BysciyDBahUtWU55RjNPUlFjMTQ/view
> 
> 
> Some additional info with the drwaing:
>   - Each linux firewall server handles top ~5G traffic. After that we
>     build a new firewall cluster and new servers are connected to
>     that one.
> 
>   - The linux firewall is gateway for the vlan.
> 
>   - As you might know: LVS is internally also a DNAT, so bassicly we do
>     a double DNAT.
> 
>   - The green and blue line indicates how the traffic is routed
> 
>   - Linux firewalls is physical hardware, but all the servers
>     (including the squid hosts) are linux vservers (like LXC containers
>     or docker containers).
> 
> 
> Hopes this clears up our setup and where I run into problems with squid3.
> 
> Regards,
> 
> --
> Henry Paulissen - PD0OM
> mailto:henry at nitronetworks.nl - Phone: +31-(0)6-115.305.64 Linux/Unix System 
> Engineer
> 
> On 30-09-16 00:35, Eliezer Croitoru wrote:
>> Hey Henry,
>>
>> I want to emulate the setup to understand the complication with a FULL linux based setup here on my local testing grounds.
>> Can you give more details on the networks in the form of subnets and VLAN numbers?
>> What is not clear to me is: Who is doing the DNAT?
>> Also, if you have not used tproxy and intercept on the PROXY machine you should re-think the whole logic of the system first before deciding on the next step.
>> There are systems which needs redesign when moving from Squid 2 to 3 or 4.
>> When I and you will have the right understanding of the scenario I believe we can find the right path if this is not already there.
>>
>> Let me know if these( the diagrams..):
>> http://wiki.squid-cache.org/EliezerCroitoru/Drafts/MwanLB#Intoduction
>> _
>> to_MultiWAN_LoadBalancing
>> http://wiki.squid-cache.org/ConfigExamples/UbuntuTproxy4Wccp2
>>
>> Make any sense to you so we can find the right words to fill the gaps in the situation.
>> Once I will have the right picture I would probably have enough information to draw some picture in VISIO and move forward to the Systems table.
>>
>> Eliezer
>>
>> ----
>> Eliezer Croitoru
>> Linux System Administrator
>> Mobile+WhatsApp: +972-5-28704261
>> Email: mailto:eliezer at ngtech.co.il
>>
>>
>> -----Original Message-----
>> From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org]
>> On Behalf Of Henry Paulissen
>> Sent: Thursday, September 29, 2016 5:40 PM
>> To: mailto:squid-users at lists.squid-cache.org
>> Subject: [squid-users] External nat'ed transparent proxy
>>
>> Hi all,
>>
>> In the company I work for we are currently using squid v2 proxies in transparent mode to intercept traffic from servers to the outside (access control).
>>
>> The technical solution for this is roughly as follows:
>> [server] -> [gateway] -> [firewall]
>>                               |
>>     ----------- DNAT ---------
>>    v
>> [squid]  -> [gateway] -> [firewall] -> [internet router]
>>
>> Our firewalls (who live between the vlan gateway and internet router), DNAT the traffic towards separate squid proxies (who are in a lvs cluster). These squid proxies are in their own vlan with special permissions to allow unrestricted port 80 outbound, etc, etc...
>>
>> Because squid v2 is becoming more and more obsolete we are looking at upgrading it towards squid v3.
>>
>> From what I read in the manuals, transparent mode is replaced by intercept (and tproxy) mode. But both dont seem to be fully backward complaint with the v2 transparent mode.
>>
>> The old trasparent mode allowed us to just dnat traffic towards the squid host without the need for the client to be aware of this. For example, the old style accepted 'GET / HTTP/1.1' (without full URL in the GET request and looking at the Host header for the destination).
>>
>> The new intercept mode comes close to this behavior, but instead of 
>> remotly dnat, it wants us to next-hop it towards the squid proxy and 
>> redirect it locally. This is problematic for us as firewall and squid 
>> proxy dont live in the same vlan, so next-hop should be the router to 
>> that vlan (and forgetting about the path back to the server).
>> Secondly, and not less blocking, we use vservers (predecessor to 
>> linux containers
>> lxc) as such, we dont have any promiscuous interfaces rights within the container.
>>
>>
>> Is there still a option to emulate normal 'regular´ style squid (as without any listen options) but instead accepting the URI path in the GET request and looking at the Host header for the destination? (lets call it passthrough mode?).
>>
>> Or, is there in squid3 a new and better way to facilitate larger setups, with the knowledge the server, firewall and squids are all in different vlans (and no, we dont have Cisco firewalls in between them ;-)).
>>
>>
>> Thanks in advance,
>>
>> --
>> Henry Paulissen - PD0OM
>> mailto:henry at nitronetworks.nl - Phone: +31-(0)6-115.305.64 Linux/Unix System 
>> Engineer
>>
>>
> 
> 
> 