Exchange 2013 Clients not able to connect using Outlook Anywhere
I would like to share an issue occurring frequently with Exchange 2013 deployments. The behavior being reported by users is their Outlook clients cannot connect to any of the Exchange Web Services (OWA, EWS). Prior to exhibiting the behavior, there were no reported configuration changes made to the web services by the Exchange Admins, so what changed? The first place that you need to look at is that Exchange back end web site certificate bindings on port 444. Normally, if you check the IIS logs and the HTTP proxy logs you can see that you get Status code 500 when the connection proxy to the Exchange 2013 back end website. Another symptom that you see is the Exchange PowerShell console won't open a session to the server. The reason that this issue happens is due to the way connection flow happens in Exchange 2013. Below you can see the data flow when outlook client connects to exchange 2013 mailbox, and the explanation of the connection. I will cover the normal flow first and then we’ll go over why this breaks when missing or having a third party certificate bound to 444 on the Exchange Back End Web Site.
Exchange 2013 Mailbox
Outlook >>>>> 2013 CAS DWS:443
2013 CAS >>>> 2013 MBX:444
2013 CAS <<<< 2013 MBX:444
Outlook <<<<< 2013 CAS DWS:443
Step by Step Connection Flow
IIS authenticates the request and Microsoft.Exchange.FrontEndHttpProxy.dll running in the MSExchangeRpcProxyAppPool. HTTP Proxy determines the Mailbox Anchor, accessing Active Directory to look up the user, then maps to the user object that contains the mailbox GUID. As soon as we obtain the GUID, HTTP Proxy refers to Active Manager to locate the database where the mailbox resides. At this point, HTTP Proxy then builds the URL to the mailbox server that contains the user's mailbox GUID@Domain.com.
HTTP Proxy takes the authenticated request and creates a serialized blob (that contains SID, user groups, and more. Then a new header is attached that contains the Client Access server name, in order to use CSC (serialized access token authentication, also known as server to server authentication) and sends this new http request URL to the Mailbox server over SSL on port 444. The new URL contains the Mailbox server name so that the Mailbox server knows the request belongs to this server.
The request is sent to the Mailbox server's RPC Virtual Directory that is running in the MSExchangeRpcProxyAppPool by using by default, integrated authentication.
RPCProxy.dll on the Mailbox server is responsible for extracting the RPC packet from the tunnel and forwarding it to the local RPC Client Access Service on Port 6001.
Now that we went over the connection flow when a mailbox is located on an Exchange 2013 mailbox server, we can cover why this breaks and why is so important of not changing or having a third party certificate bound to port 444 on the Exchange Back End Web Site. When the client connects to the CAS, and HTTP Proxy determines where the mailbox is located, it builds an URL to the destination mailbox server on port 444. The URL is built using the hostname. Once we have the URL we try to connect on secure port 444 using SSL. In the process of the SSL handshake there is several validation task that have to take place before the connection is allowed. If the name being used on the URL is not in the certificate the request will fail and your client can’t connect.
But I see 200 status codes in the IIS logs…
The reason you ‘ll see the 200 status codes on IIS is due to the initial communication that is occurring on port 443 and on the Default website. Where the process breaks is during the second part of connecting to the mailbox server on port 444. If you look closely at the HTTP proxy logs, you will have two columns one for HTTP status and one for backend status. In that log you can see the 200 on the first hop but when passing it to the mailbox server on port 444 it fails with the 500 error. Another log that you can check is the IIS log for W3SVC2. In this folder you have the logs for all traffic going thru the Exchange Back End web site.
Screenshot 1 shows wrong binding configuration
Screen Shot 2 of Exchange Shell Error when certificate is misconfigured
Screenshot 3 after editing (corrected) the Exchange Back End Certificate Binding
Screenshot 4
Screenshot 5 and 6 of OWA behavior when certificate binding incorrect on the Exchange Back End Web Site
After logging in you get a blank page
Conclusion
The main point of this article is that if you have incorrect certificate bindings on the Exchange Back End web site, it can cause the web services on Exchange 2013 server not to work properly. This incorrect certificate binding will break the connection flow, causing clients to have a bad experience. Always make sure the Exchange Back End certificate bindings for 444 always is configured to use the self-signed certificate.
Comments
- Anonymous
January 01, 2003
Thank you Mikhael - Anonymous
January 01, 2003
Thank you Mikhael Ortiz. - Anonymous
August 05, 2014
Thanks a lot for sharing - Anonymous
August 05, 2014
Thankyou Miguel.. - Anonymous
November 23, 2014
Thanks a lot Mikhael Ortiz . very simple way you have explained. :-) - Anonymous
May 14, 2015
Awesome Work Miguel!! - Anonymous
June 03, 2015
hello Mikhael. when default installation have finished Exchange Backend site choose certificate with server name which issued by domain ca authority. may we change it to self signed certificate or leave? - Anonymous
June 03, 2015
As long as the third party certificate contains the hostname is ok. If the third party certificate do not contain the hostname, then you need to revert it to the self signed certificate. Let me know if you have any other questions. - Anonymous
June 08, 2015
I did revert to the self signed certificate but now the Default Website won't start, claiming another site may be using the port. But all the bindings look correct, including the Microsoft Exchange certificate? Also, see this in the system events: The World Wide Web Publishing Service (WWW Service) did not register the URL prefix https://127.0.0.1:443/ecp for site 1. The necessary network binding may already be in use. The site has been disabled. The data field contains the error number. EAC and EMC still both unavailable? - Anonymous
June 09, 2015
@ Brian. If you only have one IP address bound to the NIC then you cannot specify the IP in the Default Web Site and the Backend Website. Instead use "unassigned" for port 444, and 443. Let me know if there are any other questions. - Anonymous
February 11, 2016
In our environment, we see the Exchange Backend website's SSL certificate is binding to "Microsoft Exchange" but I see when we try to access the RPC Proxy url "https://mailboxserverhostname:444/rpc/rpcproxy.dll?mailboxserverhostname:6001", I get an error message, certificate not trusted. we have seen this self signed certificate is not existing in Trusted CA.The only impact we have seen is RPC self test probes failing..
when the SSL binding is same for the entire site in IIS - "Exchange Backend", why do we have only this outlook self test probes failing or how come everything else working fine?
can you help me understand this ? - Anonymous
March 21, 2016
I have read the Blog and maybe I am doing something wrong but I still can't get my 2007 outlook clients to connect to my 2013 Exchange. Does anyone have clear steps I can use to fix my issue? If so please send them to thad.ts1@gmail.com Any help would be greatly appreciated. - Anonymous
October 19, 2016
It is neccessary to assign the IIS service to the self-signed certificate which I want to bind on 444 on the Exchange Back End?Currently the self-signed ceritificate is assigned to the SMTP service only and on the binding we have different certificate. I want to assign the self-signed certificate to the binding and I wonder if I have to assign the IIS service to the self-signed first and after that change the binding??Thanks. - Anonymous
November 08, 2016
Well written, thanks Mikhael.I would only add that I discovered this error by checking the Event Viewer on the server, where I discovered a load of Errors in the Windows Logs > System list, The error is: Event ID 15021 — HTTP Service SSL Availability. The exact text of the error on my server was: An error occurred while using SSL configuration for endpoint 0.0.0.0:444. - Anonymous
September 24, 2017
Thanks in favor of sharing such a good thinking, post is nice, thats why i have read it fully