[squid-users] Squid 3.4.1 as a reverse Proxy for Exchange 2016 - Problems with Exchange Active Sync

Wed Feb 10 16:42:17 UTC 2016

On 11/02/2016 3:40 a.m., Philipp Fischer wrote:
> Hi,
> 
> i´m trying to get squid working as reverse Proxy for Exchange 2016.
> 
> Actually  it works quite smooth for OWA but fails again and again
> when trying to connecto to the EAS Site of my Exchange.
> 
> From the IIS-Logs of the mailserver i can see 401 replys two or three
> times and then squid sends the user credentials which results in a
> 200 message.
> 
> When using EAS the connect (from Exchange remote connectivty
> analyzer) connects to the autodiscover site first – which produces
> about three 401´s and then to the eas-site – again about two or three
> 401´s.
> 
> Most times after the 5th 401 reply (not authenticated) squid cuts off
> the Connection – is there any way to push squid to do more retries
> after a 401 error?

I think you may have the wrong idea here. 401 is not an error, and it
not a message that matters to Squid. It is an instruction from the web
service to the client. Suqid shoul dmost definitely *not* retry when it
sees one.

Squid job in this case is simply to relay the 401 to the client. So that
the client can supply the necessary credentials on its next (repeated)
request.

Your cache_peer lines already contains "login=PASSTHRU
connection-auth=on" which is correct for handling any type of 401 login
that the server might want to use. Evrything else is a problem directly
between the IIS and the client browser.

>
> Here´s my IIS-Log:
> 
> 2016-02-10 13:18:56 10.89.5.3 OPTIONS /Autodiscover/Autodiscover.xml &CorrelationID=<empty>;&ClientId=PBUQLXUG4EOZTNG08F3BGG&cafeReqId=f7e3c699-96b5-4e79-8a54-03334a9fe7fe; 443 - 10.89.5.248 Microsoft-Server-ActiveSync/12.0+(TestExchangeConnectivity.com) - 401 0 0 78
> 
> 2016-02-10 13:18:56 10.89.5.3 POST /Autodiscover/Autodiscover.xml &CorrelationID=<empty>;&ClientId=LBQ9QZKE0YCFS3BZUOYG&cafeReqId=392f5e78-5272-4ed1-a062-c1706e341dbf; 443 - 10.89.5.248 Microsoft-Server-ActiveSync/12.0+(TestExchangeConnectivity.com) - 401 0 0 0
> 
> 2016-02-10 13:19:04 10.89.5.3 POST /Autodiscover/Autodiscover.xml &CorrelationID=<empty>;&ClientId=XTSRHRKWV0ETDZKCRLXO6W&cafeReqId=684a8948-eeb9-401f-a9bf-61d2581f138f; 443 DOMAIN\test 10.89.5.248 Microsoft-Server-ActiveSync/12.0+(TestExchangeConnectivity.com) - 200 0 0 7515
> 
> 2016-02-10 13:19:05 10.89.5.3 OPTIONS /Microsoft-Server-ActiveSync/default.eas &CorrelationID=<empty>;&ClientId=POVVU5U8Z0SDIKZWQEPRNA&cafeReqId=109adc1a-93f9-4491-aadc-b615e5b7c36a; 443 - 10.89.5.248 Microsoft-Server-ActiveSync/12.0+(TestExchangeConnectivity.com) - 401 2 5 0
> 
> Line 390924: 2016-02-10 13:19:07 10.89.5.3 OPTIONS /Microsoft-Server-ActiveSync/default.eas &CorrelationID=<empty>;&ClientId=YDXPIAVIDEYCERPKPSRQ&cafeReqId=402020a7-2bd9-4540-9050-b6dd9e34a136; 443 - 10.89.5.248 Microsoft-Server-ActiveSync/12.0+(TestExchangeConnectivity.com) - 401 2 5 124
> 
> Line 390929: 2016-02-10 13:19:07 10.89.5.3 OPTIONS /Microsoft-Server-ActiveSync/default.eas &CorrelationID=<empty>;&ClientId=PTIZMVB6NUOH1AJ1MGWWA&cafeReqId=2b329cfa-a318-4102-bb5c-3a7652e84d5d; 443 DOMAIN\test 10.89.5.248 Microsoft-Server-ActiveSync/12.0+(TestExchangeConnectivity.com) - 200 0 0 390
> 
> Line 390934: 2016-02-10 13:19:07 10.89.5.3 POST /Microsoft-Server-ActiveSync/default.eas Cmd=FolderSync&User=test&DeviceId=1797657923&DeviceType=TestActiveSyncConnectivity&CorrelationID=<empty>;&ClientId=QC284CQDGK2JMD9TA6X6G&cafeReqId=d9ceafa7-9767-4552-bb34-2444c6435a10; 443 - 10.89.5.248 Microsoft-Server-ActiveSync/12.0+(TestExchangeConnectivity.com) - 401 2 5 0
> 

This looks normal by itself. It is simply the traffic making it through
Squid. NO problems are visible at that end.

> And here´s my Squid.conf:
> 
> # This file is automatically generated by pfSense
> # Do not edit manually !
> 
> icp_port 0
> dns_v4_first on
> pid_filename /var/run/squid/squid.pid
> cache_effective_user proxy
> cache_effective_group proxy
> error_default_language en
> icon_directory /usr/pbi/squid-amd64/local/etc/squid/icons
> visible_hostname localhost

Very bad idea to use "localhost" as your publicy visible domain name.

Recent Squid can auto-detect hostname quite well, and if that actually
has a problem, then use a resolvable FQDN.

Ubuntu found out the hard way some years back that there are ISP out
there also setting their proxies names to "localhost". Which results in
one end or the other simply dropping the apparently looped traffic
in-transit.

> cache_mgr admin at localhost

Squid error pages instruct your users to email their problem reports to
this addres. Squid itself will try to email it crash reports if/when
they happen.

> access_log /var/squid/logs/access.log
> cache_log /var/squid/logs/cache.log
> cache_store_log none
> netdb_filename /var/squid/logs/netdb.state
> pinger_enable on
> pinger_program /usr/pbi/squid-amd64/local/libexec/squid/pinger
> 
> logfile_rotate 10
> debug_options rotate=10

No need for that second line to exist unless you want the cache.log and
access.log rotation numbers to be different.

The above two sections of lines all look like defaults anyway. If so,
then you dont even need to configure any of them.

> shutdown_lifetime 3 seconds

NP: if you want short shutdown timeout to avoid problems then I highly
recommend upgrading to one of the latest Squid releases (3.5.10+).

> uri_whitespace strip
> 
> acl dynamic urlpath_regex cgi-bin \?
> cache deny dynamic

The above is unnecessary. Squid correctly handles dynamic content and
your refresh_pattern is already setup for the rare cases where there are
problems.

> 
> cache_mem 64 MB
> maximum_object_size_in_memory 256 KB
> memory_replacement_policy heap GDSF
> cache_replacement_policy heap LFUDA
> cache_dir ufs /var/squid/cache 100 16 256

These ...

> forwarded_for on
> minimum_object_size 0 KB
> maximum_object_size 4 MB
> offline_mode off
> cache_swap_low 90
> cache_swap_high 95

... are all defaults. No need to specify them.

> cache allow all

If you remove the above cache deny lines then you can also remove this one.

> # Add any of your own refresh_pattern entries above these.
> refresh_pattern ^ftp:    1440  20%  10080
> refresh_pattern ^gopher:  1440  0%  1440
> refresh_pattern -i (/cgi-bin/|\?) 0  0%  0
> refresh_pattern .    0  20%  4320
> 
> 
> #Remote proxies
> 
> 
> # Setup some default acls
> # From 3.2 further configuration cleanups have been done to make things easier and safer. The manager, localhost, and to_localhost ACL definitions are now built-in.
> # acl localhost src 127.0.0.1/32
> acl allsrc src all

The point of these ACLs being "built-in" is that you can *use* them
anywhere in the config without having t have thee "acl ..." lines
definign them manually.

So consider what is the point of the "allsrc" ACL? its identical to
"all" except that Squid has to use extra memory for the one with custom
alternative name.

I recommend replacing all uses of "allsrc" with "all" and removing the
above.

> acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 8080   1025-65535 
> acl sslports port 443 563 8080 
> 
> # From 3.2 further configuration cleanups have been done to make things easier and safer. The manager, localhost, and to_localhost ACL definitions are now built-in.
> #acl manager proto cache_object
> 

... same goes for this.

> acl purge method PURGE
> acl connect method CONNECT
> 
> # Define protocols used for redirects
> acl HTTP proto HTTP
> acl HTTPS proto HTTPS
> http_access allow manager localhost
> 
> http_access deny manager
> http_access allow purge localhost
> http_access deny purge

NP: current best practice is to put the above lines underneath the two
below. So the Safe_ports and CONNECT protections (which are DoS
protection) are applied before the slower and more resource hungry
manager and purge rules.

> http_access deny !safeports
> http_access deny CONNECT !sslports
> 
> # Always allow localhost connections
> # From 3.2 further configuration cleanups have been done to make things easier and safer.
> # The manager, localhost, and to_localhost ACL definitions are now built-in.

That above comment has no relevance here. This part of squid.conf is
about *using* the localhost ACL.

> # http_access allow localhost
> 
> request_body_max_size 0 KB

Another default.

> delay_pools 1
> delay_class 1 2
> delay_parameters 1 -1/-1 -1/-1
> delay_initial_bucket_level 100
> delay_access 1 allow allsrc
> 

Um. This is bad. Forces Squid to do a lot of delayPools traffic
accounting memory allocations and calculations. In an attempt to *not*
slow the traffic down. Very counter-productive.

> # Reverse Proxy settings
> http_port 10.0.0.3:80 accel defaultsite=mail.DOMAIN.de vhost
> https_port 10.0.0.3:443 accel cert=/usr/pbi/squid-amd64/local/etc/squid/56b21e54ee18f.crt key=/usr/pbi/squid-amd64/local/etc/squid/56b21e54ee18f.key  defaultsite=mail.DOMAIN.de vhost
> cache_peer 10.89.5.3 parent 443 0 proxy-only no-query no-digest originserver login=PASSTHRU connection-auth=on ssl sslflags=DONT_VERIFY_PEER front-end-https=on name=OWA_HOST_443_1_pfs
> cache_peer 10.89.5.3 parent 80 0 proxy-only no-query no-digest originserver login=PASSTHRU connection-auth=on name=OWA_HOST_80_1_pfs
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/owa.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/exchange.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/public.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/exchweb.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/ecp.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/OAB.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/Microsoft-Server-ActiveSync.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/rpc/rpcproxy.dll.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/rpcwithcert/rpcproxy.dll.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/mapi.*$
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/EWS.*$
> acl OWA_URI_pfs url_regex -i ^http://mail.DOMAIN.de/AutoDiscover/AutoDiscover.xml
> acl OWA_URI_pfs url_regex -i ^https://mail.DOMAIN.de/AutoDiscover/AutoDiscover.xml
> acl OWA_URI_pfs url_regex -i ^http://autodiscover.DOMAIN.de/AutoDiscover/AutoDiscover.xml
> acl OWA_URI_pfs url_regex -i ^https://autodiscover.DOMAIN.de/AutoDiscover/AutoDiscover.xml
> cache_peer_access OWA_HOST_443_1_pfs allow OWA_URI_pfs
> cache_peer_access OWA_HOST_80_1_pfs allow OWA_URI_pfs

Your OWA ACL has a mix of http:// and https:// URLs that it matches.
With Exchange and IIS its usually not a good idea to mix those up, and
teh serevr can auto-redirect http:// URLs to teh https:// variant and
loop if Squid sends to the wrong one. Or Squid can inadvertently publish
what teh server things is secured information over the port-80 client
connections.

Others have found these setups work much better if you lock the traffic
to only going to the cache_peer with same port type as it arrived to
Squid over.

like so:

 acl HTTPS proto HTTPS
 acl HTTP proto HTTP

 cache_peer_access OWA_HOST_443_1_pfs allow HTTPS OWA_URI_pfs
 cache_peer_access OWA_HOST_80_1_pfs allow HTTP OWA_URI_pfs

Where the OWA_URI_pfs ACL does not contains the "http(s)://" portion of
the URLs.

OR, if you want to strictly require only those path+scheme combos, use
two ACLs - one for each peer. With https:// entries in one, and http://
in the other.

> cache_peer_access OWA_HOST_443_1_pfs deny allsrc
> cache_peer_access OWA_HOST_80_1_pfs deny allsrc
> never_direct allow OWA_URI_pfs
> http_access allow OWA_URI_pfs
> 
> 
> # Custom options before auth
> 
> 
> # Setup allowed ACLs
> # Default block all to be sure
> http_access deny allsrc
> 
> I also tried to Change the squid.conf to a example config from the faq´s – which had the same effect as described above.
> 
> Any ideas anybody?

Amos