Replacing Windows Server 2016 TP5 AD FS with RTM Experience
Senior Microsoft Premier Field Engineer Frank Plawetzki guides us though his recent Windows 2016 AD FS upgrade from TP5 to RTM
My AD FS lab environment is hosted in Azure which I use for testing AD FS claims based authentication with Exchange OWA and ECP. For details on how to deploy this claims based solution please follow the documented process.
Since my lab environment was still running the technical preview 5 (TP5) of Windows Server 2016 for both WAP and AD FS roles, I needed to act before March 2017, since from then onwards the TP5 would not function anymore. Besides, for troubleshooting I also wanted to move to the RTM version of Windows server 2016. As of this writing this was version 1607, build 14393.693:
The migration from TP5 to RTM was not as smooth as I expected and with this post I want to describe the issues I have encountered and how I managed to solve them.
In my example the hostnames of the TP5 machines were ADFS05 and WAP05 whereas the hostnames of the new Windows 2016 RTM servers are ADFS06 and WAP06.
Throughout the past, e.g. when migrating from TP4 to TP5 the high-level process has always been:
1. Install ADFS06 and WAP06 servers
2. Join ADFS06 server to the domain
3. Import your public SSL service communication certificate into ADFS06 server local machine store
4. Join ADFS06 server to the existing ADFS farm (by doing this, the new ADFS server will copy the configuration settings over)
5. Changing ADFS05 server to a secondary role and ADFS06 server to primary ADFS server
6. Join WAP06 to the ADFS farm
7. Remove WAP05 from the ADFS farm
8. Un-install ADFS05 server (since Remove-ADFSFarmNode is deprecated)
Configuring the AD FS Role
When you perform step no. 4 from above in PowerShell, it should be something like:
Add-AdfsFarmNode -CertificateThumbprint:"5F91EC6EE19CA40D8B6B14878127E8FCB19C8587" -GroupServiceAccountIdentifier:"CONTOSO\GMAadfs`$" -PrimaryComputerName:"ADFS05.CONTOSO.COM" -OverwriteConfiguration
The corresponding output was:
WARNING: A machine restart is required to complete ADFS service configuration. For more information, see: https://go.microsoft.com/fwlink/?LinkId=798725
WARNING: The SSL certificate subject alternative names do not support host name 'certauth.sts2.frankpllabs.info'.Configuring certificate authentication binding on port '443' and hostname 'sts2.frankpllabs.info'.WARNING: The SSL certificate does not contain all UPN suffix values that exist in the enterprise. Users with UPN suffix values not represented in the certificate will not be able to Workplace-Join their devices. For more information, see https://go.microsoft.com/fwlink/?LinkId=311954.)
Step no. 5 was achieved by running:
On ADFS06: Set-AdfsSyncProperties -Role PrimaryComputer andOn ADFS05: Set-AdfsSyncProperties -Role SecondaryComputer -PrimaryComputerName ADFS06.contoso.com
After that I had to change the DNS records for the STS2 in my on-premises environment to point to ADFS06.
So, for the ADFS part, all seemed good.
Configuration of the WAP Role (and let the fun begin)
Next was the installation of WAP06:
I also imported the public SSL service communication certificate into the local machine store.
The PowerShell cmdlet for installing the WAP role was:
Install-WebApplicationProxy -FederationServiceTrustCredential System.Management.Automation.PSCredential -CertificateThumbprint '5F91EC6EE19CA40D8B6B14878127E8FCB19C8587' -FederationServiceName 'STS2.frankpllabs.info'
At this point, the cmdlet failed unexpectedly with the error message:
“Unable to retrieve proxy configuration data from the Federation Server.”
In the same moment, the ADFS server was throwing this error message in the ADFS event log:
Log Name: AD FS/Admin
Source: AD FS
Date: 17.01.2017 15:36:02
Event ID: 276
Task Category: None
Level: Error
Keywords: AD FS
User: CONTOSO\GMAadfs$
Computer: ADFS06.contoso.com
Description:
The federation server proxy was not able to authenticate to the Federation Service.
User Action
Ensure that the proxy is trusted by the Federation Service. To do this, log on to the proxy computer with the host name that is identified in the certificate subject name and re-establish trust between the proxy and the Federation Service using the Install-WebApplicationProxy cmdlet.
So obviously the WAP server could not establish a trusted connection to the ADFS server using its certificate. My co-worker Ian has written an extensive blog post about issues surrounding trust issues between WAP and ADFS here: Understanding and fixing Proxy Trust CTL Issues with AD FS 2012 R2 and Web Application Proxy
Following those troubleshooting procedures, I used the PowerShell script attached to the blog post and the output was:
.\Check-ADFS.ps1
1 – Checking http.sys certificate bindings for potential issues
IP:port : 0.0.0.0:5986
Hostname:port : localhost:443
Hostname:port : sts2.frankpllabs.info:443
WARNING: ADFS Service binding does not have CTL Store Name set to AdfsTrustedDevice
Hostname:port : EnterpriseRegistration.frankpllabs.info:443
2 – Checking AdfsTrustedDevices cert store for non-self signed certificates
Check Passed: No non-self signed certs present in AdfsTrustedDevices cert store
3 – Checking AdfsTrustedDevices cert store is in sync with ADFS Proxy Trust config
Check Passed: No mismatched certs found. CTL is in sync with DB content
So, to check that further, I checked the SSL certificate bindings:
netsh http show sslcert
SSL Certificate bindings:
-------------------------
IP:port : 0.0.0.0:5986
Certificate Hash : 0b50fdfd4faf12a2f97f412ad4485148e0933243
Application ID : {afebb9ad-9b97-4a91-9ab5-daf4d59122f6}
Certificate Store Name : (null)
Verify Client Certificate Revocation : Disabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check : Disabled
Revocation Freshness Time : 0
URL Retrieval Timeout : 0
Ctl Identifier : (null)
Ctl Store Name : (null)
DS Mapper Usage : Disabled
Negotiate Client Certificate : Disabled
Reject Connections : Disabled
Hostname:port : localhost:443
Certificate Hash : 5f91ec6ee19ca40d8b6b14878127e8fcb19c8587
Application ID : {5d89a20c-beab-4389-9447-324788eb944a}
Certificate Store Name : MY
Verify Client Certificate Revocation : Enabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check : Enabled
Revocation Freshness Time : 0
URL Retrieval Timeout : 0
Ctl Identifier : (null)
Ctl Store Name : AdfsTrustedDevices
DS Mapper Usage : Disabled
Negotiate Client Certificate : Disabled
Reject Connections : Disabled
Hostname:port : sts2.frankpllabs.info:443
Certificate Hash : 5f91ec6ee19ca40d8b6b14878127e8fcb19c8587
Application ID : {5d89a20c-beab-4389-9447-324788eb944a}
Certificate Store Name : MY
Verify Client Certificate Revocation : Enabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check : Enabled
Revocation Freshness Time : 0
URL Retrieval Timeout : 0
Ctl Identifier : (null)
Ctl Store Name : (null)
DS Mapper Usage : Disabled
Negotiate Client Certificate : Enabled
Reject Connections : Disabled
Hostname:port : EnterpriseRegistration.frankpllabs.info:443
Certificate Hash : 5f91ec6ee19ca40d8b6b14878127e8fcb19c8587
Application ID : {5d89a20c-beab-4389-9447-324788eb944a}
Certificate Store Name : MY
Verify Client Certificate Revocation : Enabled
Verify Revocation Using Cached Client Certificate Only : Disabled
Usage Check : Enabled
Revocation Freshness Time : 0
URL Retrieval Timeout : 0
Ctl Identifier : (null)
Ctl Store Name : AdfsTrustedDevices
DS Mapper Usage : Disabled
Negotiate Client Certificate : Disabled
Reject Connections : Disabled
As you can see above, the highlighted CTL Store name indeed was empty which is not good.
The blog post mentions to reset the ADFS SSL certificate via Set-AdfsSSLCertificate, in order to fix this. Therefor I executed the following command which also errored out:
Set-AdfsSslCertificate -Thumbprint 5F91EC6EE19CA40D8B6B14878127E8FCB19C8587
Set-AdfsSslCertificate : PS0159: The operation is not supported at the current Farm Behavior Level '2'. Raise the farm to at least version '3' before retrying.
At line:1 char:1
+ Set-AdfsSslCertificate -Thumbprint 5F91EC6EE19CA40D8B6B14878127E8FCB1 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Set-AdfsSslCertificate], DisplayableArgumentException
+ FullyQualifiedErrorId : Microsoft.IdentityServer.Diagnostics.Exceptions.DisplayableArgumentException,Microsoft.I
dentityServer.Management.Commands.SetSslCertificateCommand
So, the current ADFS farm level still was a mixed level (2) which says the farm includes Windows 2012 R2 and Windows 2016 AD FS servers.
My first attempt to raise the ADFS farm level via Invoke-AdfsFarmBehaviorLevelRaise failed, because the cmdlet stated that the farm still contained Windows 2012 R2 AD FS servers. This was also unexpected.
Since every time I had removed an AD FS server by gracefully uninstalling the server the status was then checked with Get-AdfsFarmInformation which showed all the old ADFS servers were all long gone.
Anyway, I managed to remove them via "Set-AdfsFarmInformation -RemoveNode "adfs02.contoso.com" and repeated this command for every old AD FS hostname.
Now, the farm information only included the new remaining ADFS06 server.
So, I finally arrived at the point to raise the ADFS farm level via: Invoke-AdfsFarmBehaviorLevelRaise
The resulting output was:
WARNING: AD FS Server: ADFS06.contoso.com, Warning: The persistent SSO lifetime has been upgraded from '60480' mins to
'129600' mins.
The device usage window has been upgraded from '7' days to '14' days.
WARNING: Failed to add service account 'CONTOSO\GMAadfs$' to the Enterprise Key Admins group. Add the service account
to the Enterprise Key Admins group.
In my understanding the last warning can be safely ignored and will vanish as soon as a Windows 2016 DC is in place.
So, at this point, I was able to run the intended: Set-AdfsSslCertificate -Thumbprint 5F91EC6EE19CA40D8B6B14878127E8FCB19C8587.
The result was satisfying:
As the command completed successfully, I thought that the migration procedure was finished by then, but I was wrong.
When I tested the ADFS and WAP for functionality via accessing the STS URL https://sts2.frankpllabs.info/adfs/ls/idpinitiatedsignon.aspx I realized, that the WAP service was not running.
On top of that, the ADFS service was running, but it could not register its endpoints, because ADFS event log showed this error:
Log Name: AD FS/Admin
Source: AD FS
Date: 24.01.2017 13:05:56
Event ID: 102
Task Category: None
Level: Error
Keywords: AD FS
User: CONTOSO\GMAadfs$
Computer: ADFS06.contoso.com
Description:
There was an error in enabling endpoints of Federation Service. Fix configuration errors using PowerShell cmdlets and restart the Federation Service.
Additional Data
Exception details:
System.ServiceModel.AddressAccessDeniedException: HTTP could not register URL https://+:49443/adfs/services/trust/2005/certificatetransport/. Your process does not have access rights to this namespace (see https://go.microsoft.com/fwlink/?LinkId=70353 for details). ---> System.Net.HttpListenerException: Access is denied
at System.Net.HttpListener.AddAllPrefixes()
at System.Net.HttpListener.Start()
at System.ServiceModel.Channels.SharedHttpTransportManager.OnOpen()
--- End of inner exception stack trace ---
at System.ServiceModel.Channels.SharedHttpTransportManager.OnOpen()
at System.ServiceModel.Channels.TransportManager.Open(TransportChannelListener channelListener)
at System.ServiceModel.Channels.TransportManagerContainer.Open(SelectTransportManagersCallback selectTransportManagerCallback)
at System.ServiceModel.Channels.TransportChannelListener.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.HttpChannelListener`1.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Dispatcher.ChannelDispatcher.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at Microsoft.IdentityServer.ServiceHost.STSService.StartSTSService(ServiceHostManager serviceHostManager, ServiceState serviceState)
This again was not expected. This issue occurred because there is an issue with HTTP namespace delegations which need to be fixed via NETSH (thanks to my co-worker Bartosz).
This is described in the following post: System.ServiceModel.AddressAccessDeniedException: HTTP could not register URL
So, the fix for the issue in my case was:
netsh http add urlacl url=https://+:49443/ user=contoso\GMAadfs$
Finally, I restarted the AD FS service and WAP services and verified via the event logs that both roles started without issues and my workloads using AD FS claims based authentication started working again.
Published by MSPFE editor Rhoderick Milne