Troubleshooting SSL related issues with IIS

Article
09/05/2007

In this post I am going to discuss troubleshooting SSL server certificate related issues in IIS. Please remember I am no SSL expert...I am an IIS Asp.Net guy and will discuss issues with regard to SSL with IIS.

We have seen a lot many times that the issues could have been resolved by our customers if they were aware of some targeted troubleshooting techniques.

Here I will be walking through the steps which you can follow yourself before requiring to call Microsoft PSS.

Problem Description

The most common error associated with SSL not working for an IIS website is "The page cannot be displayed" or "Cannot find server or DNS Error".

Troubleshooting steps to be followed:

First thing first: You notice the above error when you browse to your website over SSL (HTTPS). Do you also see the same behavior when you access the website over HTTP? If yes then there is a different problem you need to address first. SSL doesn't come into picture when we access a site over HTTP.
So if you are facing the problem in both the scenarios ensure you first troubleshoot why the request is not reaching the IIS server...maybe a DNS configuration etc.

My topic today is SSL with IIS so I won't be focusing on HTTP.

We have the SSL port listed in IIS mmc as shown below. By default it's 443. You can configure it to listen on a different port.

Now, also check whether the issue is happening for all the users, i.e. External and internal users to the network. If the issue is happening only for let's say external users check whether the requests are coming over a firewall like ISA etc. At times Firewalls may block the SSL port. You may want to check the firewall settings to unblock the SSL port. Also firewalls like ISA requires certificate to be published on it as well apart from IIS server. Ensure that the certificate that we have installed on ISA is valid. In case of ISA, if you have renewed the cert on IIS server but not on ISA users will still see the old certificate and not the new certificate. Ensure that you replace the old certificate on ISA with the new certificate. Also you may want to check if the security gateway (if any) such as NFUSE (Citrix) controllers is still referencing the old certificate.

Also at times if you see a wrong certificate being displayed for your Web Site, ensure that you do not have kernel-mode SSL enabled on your web server.Although this improves the performance but I recommend to have it disabled when you have multiple SSL enabled-websites on your server. Kernel-mode SSL is only recommended if you have just one site, mostly static content and no client certificates. In all other cases you should use User-mode SSL.

Please find this link to enable/disable kernel-mode SSL. If it is set to 1 it means Kernel-mode SSL is enabled, if absent or set to 0 it means disabled and user-mode SSL will be used. Ensure that you stop and restart http service as shown below after doing the changes. No reboot is required for the above change.

~~>net stop http~~

~~>net start http~~

~~>net start w3svc~~

[There has been few concerns from people as to why this is suggested. I
do not have a proper answer to it at this point, but I have seen such
issues in some support incidents where disabling Kernel mode SSL had
resolved the above problem]

------------------------------------------------------------------------------------------------------------------------------------------------------------------

Now once you have isolated the issue to be only with HTTPS and all users let's proceed with the following steps:

Run netstat -an (or fport.exe) for IIS 5 (and netstat -ano for IIS 6) and verify whether IIS is listening on the SSL port.

When you start Internet Information Services (IIS) 6.0 on Microsoft Windows Server 2003, IIS binds to all IP addresses on the server, not just the IP addresses that are assigned to Web sites. This is because of socket pooling enabled for IIS by default.

On IIS 6.0, if the website is listening on "All unassigned" IP address or some specific IP address (like 192.134.123.209 etc) and SSL port 443, verify that Local address entry is 0.0.0.0:443 in netstat output.

> netstat -ano

Active Connections

Proto Local Address          Foreign Address        State           PID
TCP    0.0.0.0:21             0.0.0.0:0              LISTENING       2396
TCP    0.0.0.0:25             0.0.0.0:0              LISTENING       2396
TCP 0.0.0.0:80 0.0.0.0:0 LISTENING 4
TCP    0.0.0.0:81             0.0.0.0:0              LISTENING       4
TCP    0.0.0.0:83             0.0.0.0:0              LISTENING       4
    ...................

  TCP 0.0.0.0:443 0.0.0.0:0 LISTENING 4
TCP    0.0.0.0:445            0.0.0.0:0              LISTENING       4
TCP    0.0.0.0:446            0.0.0.0:0              LISTENING       4

Here the PID should show you which process is listening on that port.
Now let's say I have configured my website to run on a specific SSL port and that port is being used by a different process, then website will not start up.

Also you won't see the netstat output showing anything like this depending upon the port being used:

TCP 0.0.0.0:80 0.0.0.0:0 LISTENING 4

TCP 0.0.0.0:443 0.0.0.0:0 LISTENING 4

You can run netstat -ano again and see the PID corresponding to the process which is listening on that port. Stopping or disabling that service/process should allow Websites to be up and listening on the desired port. For IIS 5.x you can run Fport.exe to find out which process is listening on a specific port.

The first thing that I check for (strictly my personal preference) when we have issues with SSL is MACHINEKEYS. Yes, this is a very important (I prefer to call strategic) part of your checklist while dealing with SSL issues. In my personal experience it has resolved a lot of issues for our customers. By default machinekeys folder should have permission for Administrator, System and Everyone according to this KB278381. Ensure that we inherit the above permission for all child objects under the machinekeys folder, that way all the machine keys have the necessary permissions.

This is done by clicking the "Advanced" button on the security settings page, Clicking the check box "Replace permissions entries on all child objects...." and
Apply. This will propagate the settings to the contents in the folder.

Another reason for the above error ('The page cannot be displayed') can be if you do not have a private key corresponding to the SSL server certificate. You can check that by opening the IIS mmc -> <<YourWebSite>> ->Properties ->Directory Security ->Secure Communications ->View certificate ->General -> "You have a private key that corresponds to this certificate" as shown below.

If you do not see the above then SSL won't work for your website.
You can try running the following command to recover the private key from the certificate thumbprint.

To associate the certificate with its private key, we run the following command:

> certutil -repairstore my "c9 66 67 68 49 97 88 7f 05 b8 89 b7 b3 f7 37 c8 c4 da 5f 16"

Note: "c9 66 67 68 49 97 88 7f 05 b8 89 b7 b3 f7 37 c8 c4 da 5f 16" is the Thumbprint of the missing certificate. You can see this if you double-click on the .cer file, choose the Details tab and select Thumbprint.

Make sure you use the 2003 version of certutil.exe (with the associated certadm.dll and certcli.dll), or you will not have access to the repairstore command.

If the above command does not get back the private key or for some reason it still does not work you may prefer to get a new certificate from the CA.

One of the best tools that Microsoft has come up with is SSL Diagnostics to troubleshoot SSL related issues. Just download it to the IIS server and run it and if there is an error due to configuration of IIS or certificate it will show up. You can also use this tool to issue a test certificate for your website to check whether the problem was occurring because of your server certificate or some other IIS configuration related issues. Things that can be shown quickly through this tool includes on "no private key", "SSL port being used by a different process", "machine keys not having enough permission", "IP address conflicts" etc.

Another scenario can be wherein if you view the certificate, it will state that "You have a private key that corresponds to this certificate," and if you run SSL Diagnostics, you'll get the error as below:

#WARNING: You have a private key that corresponds to this certificate but CryptAcquireCertificatePrivateKey failed

When you initially go through the IIS Certificate Wizard to create a new certificate, a file is added to the MachineKeys folder. This is the private key.

If you add permissions to the new file in the MachineKeys folder before you process the pending request in the IIS certificate wizard, the certificate will work properly. If you've already gone past that point and got a new certificate, then necessary permissions should be added to the MachineKeys folder and a new certificate request must again be submitted.

Now consider a scenario wherein you are able to access the site through SSL Diagnostics tool (through 'Simulate SSL Handshake') for the same certificate but not through IE. What I mean is your SSL handshake gives no error etc in SSL Diagnostics although when you access the website through IE over SSL it still fails. In that case there could be an IE setting that you may want to check for such a behavior. Check this out KB811834.

Now one last scenario. At times we see customers coming up with requests wherein they want to use the same server certificate for multiple websites.

This is not possible. You cannot use the same certificate for multiple websites on the same server. Check this KB187504.

Now there is a workaround in IIS 6.0 Windows 2003 SP1, wherein you can use Wildcard certificates to get a similar functionality. Here is the link.

If you have a Load balanced or clustered environment you can export the same certificate to all the servers and install them on the related websites. It should work fine.

Hope this helps....let me know if there are any questions around the topic.

Cheers!!!

Comments

Anonymous
September 05, 2007
Very sweet post!
Anonymous
September 05, 2007
PingBack from http://msdnrss.thecoderblogs.com/2007/09/05/troubleshooting-ssl-related-issues-with-iis/
Anonymous
September 18, 2007
Another thing to double check is that the certificate file in the machine key folder has correctly inherited permissions. I just spent an hour and a half pouring over everything, only to find out that the system did not have full access permissions on the certificate I installed. This article saved my life though, I would have never found it otherwise.
Anonymous
September 18, 2007
Glad that this article saved you precious time. Please feel free to shoot any question if they are related to any of the topics that I work on like IIS, ASP and Asp.Net and i will try to get you an answer by some means, even if that includes me asking other people here to get your answers. Thanks!
Anonymous
December 14, 2007
Excellent Post.. I guess its a better then Microsoft explanition.:) Question: I am facing a similar problem, when I enter port 443 in the IIS website window, my website stop working and I get a message that "service is being used by another process". When i remove port 443 from IIS window, the website start 2 work fine. (But then I can't use it in the https mode.) Can I try the same solution which you have mentioned with my probelm? OR i would need to change some setting in the registry which is suggest by Microsoft ( support.microsoft.com/kb/890015 ). Best regards, Please reply to my email also. gbwebs@eim.ae
Anonymous
December 14, 2007
Hi Murtaza, thanks for your comment :-) Yes, you can follow the article that you have mentioned there but there are some checklist to ensure you are facing the same issue. -- Do you see any of those event logs mentioned in the KB article on your machine when you try to start the website related to IP Listen-only list? If yes then you can follow the KB. -- How many websites do you have in the IIS server? -- Are you running your faulting website on some specific IP address or "All Unassigned"? -- If it is IIS 6.0, do you see any other process running on SSL port, by running netstat -ano? The above pointers will give you the right action plan. Feel free to contact me if there are any questions or concerns. -Saurabh
Anonymous
January 14, 2008
The comment has been removed
Anonymous
January 14, 2008
Hi Heather, Probably my bad, but I didn't get what your question was. How about if you contact me directly through Email on Top of the page and we can take it from there. Saurabh
Anonymous
February 04, 2008
Great informational how to. This saved me a lot of time and research. I appreciate you taking the time to post this.
Anonymous
February 09, 2008
I don't understand the reasons behind recommending the disabling of Kernel Mode SSL with multiple web sites. SSL is bound to a web site based on an IP/port combination. The certificate should be chosen on that basis, and not based on any server-level attributes. With IIS 7 using Kernel Mode SSL by default it seems to further confirm that this is a good thing to use.
Anonymous
February 12, 2008
thanks so much! I was killing myself trying to figure out why my site wasn't coming up in https requests until I read your firewall comment. Added port 443 and it came right up!
Anonymous
February 17, 2008
Thank you... I simply tried to re-install the cert from verisign and as a result web server 2 didn't have the private key, and would not function. It's odd as it too the cert, and seemed like all was well but wouldn't server over 443, no errors, nothing. I went back to web1 and exported the cert with key, did the import, and all was well. Thank you so much for your post, you really helped me out!
Anonymous
February 27, 2008
If we have a SSL enabled website and a non-SSL website and when we try to access the non-SSL site over SSL port (let’s say 443) it may respond back with 400 Bad Request. Here you will still get a security alert from the other SSL enabled site. Reason being that SSL encryption/decryption occurs at http.sys level in the kernel mode. So the request host header cannot be determined when it reaches the IIS server (still encrypted), and so the request is routed to the SSL enabled site. This site will send the security alert for obvious reasons because of host header mismatch. And now since the original request header doesn’t match the host header of the SSL enabled site it will respond back with 400 Bad Request. ***Remember, even if we have multiple SSL enabled sites with different certificates, users will still get security alert because of host header mismatch. You can make the sites accessible though by providing the proper securebindings entry in the metabase but you will get wrong certificates and hence a security alert. If we want to make multiple SSL sites work (even if they have different certificates) without prompting for security alerts, we need to run them on different IP address/port combinations else if we want to use same the IP/Port we need to use Wildcard certificate on Win2k3 SP1+ environment.
Anonymous
March 18, 2008
I also do not understand your reasoning for stating not to use kernel mode SSL if you have multiple sites on the same server using SSL. Why not? trayharrison@gmail.com
Anonymous
March 18, 2008
I have highlighted the section which talks about disabling Kernel mode SSL. I am not sure why but this had resolved many prior issues for us under the scenario mentioned in the same section. Once I have an answer or findings for/against it i will post it here. If any one has an idea please share it with us. thanks.
Anonymous
May 19, 2008
Your article is very good, but my problem is little bit different. We use HTTPS for downloading. The problem is the HTTPS works intermittently. When the user is downloading the file from externally after some time the download breaks with error “Connection with the server was reset” and then if the user tries to download again the links doesn’t work “Page Cannot be displayed”. I verified in “httpsys” log file but there were no error logs. I also verified with SSL Diagnostic tool, every thing seems fine. I see one warning error with “Basic constraint” value is “Subject Type=End Entity Path Length Constraint=None” in the certificate. Is there some wrong with this parameter or some thing else? Your help is appreciated. Srividya_manam@yahoo.com
Anonymous
May 26, 2008
The comment has been removed
Anonymous
June 12, 2008
Just wanted to say thanks! My SSL private key didn't import correctly, so nothing was working even though it all appeared correct in the IIS UI. Doing the the "certutil -repairstore my" worked like a charm.
Anonymous
October 22, 2008
Hi, maybe you can make clear my situation. From client's part all work very good with https and trouble is that iis must verify authentication of client from centre of authentication (OpenLdap). It's sharepoint with forme based authentication (FBA). Trought 389 (witout SSL) all work fine, when i change from "useSSL=false" to "useSSL=true" I receive error. When I verify connection with lde.exe from IIS server to authentication ceneter all work fine with LDAPS (secured LDAP). How can I achieve same effect in sharepoint? Best regards!!! Kalvador
Anonymous
October 22, 2008
Hi Kalvador, Unfortunately i don't have expertise on SharePoint, so not sure if i can be of much help. BTW, is this Sharepoint site accessed with IE as a client that is having the problem over SSL? If yes can you try accessing this over SSL with IE and let me know if you get a security alert or not. If you get one that could be one of the probable reasons. What exactly is the error you get...can you share the error message shown or logged in somewhere...
Anonymous
October 24, 2008
Hi, Saurabh Singh, thanks for quick reply. Actually that error is between sharepoint server and OpenLDAP server (on unix like). That is Microsoft's omission. The membership provider (ASP.NET) "add name="LdapMembership" type="Microsoft.Office.Server.Security.LDAPMembershipProvider, 
 Microsoft.Office.Server, Version=12.0.0.0, Culture=neutral, 
 PublicKeyToken=71E9BCE111E9429C" server="name.domain.com" port="636" useSSL="true" useDNAttribute="false" userNameAttribute="uid" userContainer="ou=people,dc=domain,dc=com" userObjectClass="person" userFilter="(ObjectClass=person)" scope="Subtree" otherRequiredUserAttributes="uid,cn" />" maybe don't support SSL. I'm trying find solution for my occasion. Many people talk that only special connector (membership LDAP provider) can be for that solution. Any suggestions will be appreciated. Best rerards!!! Kalvador
Anonymous
November 06, 2008
HI again.... Singh, now it's work. Thanks for any support :) It was the problem with certificates BB
Anonymous
November 06, 2008
Hi BB, Great to know the issue is resolved :-)
Anonymous
December 02, 2008
Hi Saurabh; I don't know if this fits in here. We currently hosting multiple websites into two win2k3 IIS6.0 using host header, we have a load balancer on top of it. Our problem on is that we have to implement SSL cert into 1 website (website B), making it complicated we already have 1 website (website A) running SSL. website A & B has different domain. From Microsoft they ask us to use a different IP for website B, but we could not have it as a solution because in time there would be other sites from IIS that would be using SSL. Another solution was to use other port, which is also not appicable on our part. Have you encounter such scenario or may know someone who encounter such? please help. thanks. James djg_magdangal@yahoo.com
Anonymous
December 03, 2008
Hi James, If that is the case wherein you cannot use different IP addresses or ports I feel we are very limited in implementing SSL for multiple sites. Also as you mentioned these sites are having different end domains, wildcard won't be an option as well.
Anonymous
December 03, 2008
Hi Saurabh; Thanks for your immediate response. May I know if you have anyone that could help us out on this problem?as for now we have tried asking microsoft for solutions and yet they only provide us with using different IP or using other port. Anyone in here would know or encounter same scenario?
Anonymous
December 07, 2008
great post! I do not think this is correct: "You cannot use the same certificate for multiple websites on the same server." So long as you are using a separate IP or port you can reuse an SSL certificate for a separate IIS site.
Anonymous
December 09, 2008
Hi James, this is a design limitation, so I doubt anyone can help us here. Moshe, yes I should have been more specific, thanks I will add a note.
Anonymous
January 28, 2009
Thanks a lot for crunching all the certificate related issues in jst one page. this helps a ton!! Thank you
Anonymous
May 12, 2009
Thanks you, excellent article.
Anonymous
September 07, 2009
Once again thanks Saurab.
Anonymous
September 24, 2009
After installing/uninstalling over a dozen times and 2 certificates later I finally got this to work thanks to this article!!! Mega kudos to you my friend!!!
Anonymous
November 24, 2009
The comment has been removed
Anonymous
November 25, 2009
S V Saibaba, It seems the certificate authority (CA) foryour cert is not installed on your office machine. Can you tell me what exactly is the error message if you double click on the red color address bar? also see if this may help, http://blogs.msdn.com/saurabh_singh/archive/2007/11/07/you-get-a-security-alert-when-you-try-to-access-an-ssl-enabled-web-site-when-certificate-has-been-issued-by-an-internal-root-ca.aspx
Anonymous
June 01, 2010
I used the certutil to fix the private key on my EV cert - worked perfect!! Thank you!!
Anonymous
March 31, 2011
I followed all instructions but the problem continuous!!!!
Anonymous
March 31, 2011
Please check my other posts on SSL troubleshooting, they may help.
Anonymous
February 07, 2012
Great troubleshooting article. Still very relevant unfortunately :)
Anonymous
April 11, 2012
this is good meterial.. in this given detail about the permission of MACHINEKEYS

Share via

Comments

Additional resources