Two IIS_IUSRs Related Problems caused by Group Policy Overrides
Today I troubleshot an error that bubbled up in PerformancePoint Service's Dashboard Designer while attempting (and failing) to create a data connection.
The first visible error was:
Dashboard Designer
"An unexpected system error has occurred. Additional details have been logged for your administrator".
Error #2 was from the event log:
Event
ID 4
Error
52068.
Exception
details:
System.ServiceModel.ServerTooBusyException:
The HTTP service located at https://servername:port/whatever/PerformancePointService.svc
is too busy. System.Net.WebException: The remote server returned an error: (503) Server
Unavailable. at System.Net.HttpWebRequest.GetResponse() at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan
timeout)
The customer I was assisting had already created a missing PerformancePoint service proxy with PowerShell.
There were probably some other very relevant events in the system event log but I didn't make it a point to ask to see them. Instead we opened Internet Explorer and browsed to https://servername:port/whatever/PerformancePointService.svc to confirm that we saw a 503 from that angle as well. We did. No great surprise there. But it's a nice sanity check.
So what was wrong with the IIS Application pool that the PerformancePointService.svc was running in?
We opened the IIS manager and noticed that the application pool associated with this performance point service application was stopping every time we tried to use it. We could stop and start the application pool fine. But it was getting disabled every time we tried to browse to the .svc.
Normally this is a very fine time to focus on the system event log and see what the IIS related events are saying about the application pools. But since this is usually a matter of something being indirectly wrong with the account the iis application pool uses. Maybe the account isn't part of the local IIS_IUSRS group (or if IIS6, the IIS_WPG group). Maybe the account is locked out or disabled. Maybe group policy is overriding some of the user rights in the local security policy.
To test the theory that it was a problem with policies or permissions, we simply set the application pool identity into the local administrators group, ran iisreset, and confirmed that the performancepointservice.svc could be browsed to successfully in IE and the data
connection could be created just fine from Dashboard Designer.
So what went wrong? In this particular test case, the IIS appPool identity account was a member of the IIS_IUSRs group. So that's good. It needs to be there. IIS_IUSRs group arguably should have been called AppPoolIdentities group. The application pool identities need to be in
that group if you want the appPool to have a chance to spin up at all. In this case the local security policy it was clear that group policy was controlling local security policy. There were no options to make modifications. We couldn't add the IIS_IUSRS group to the user
rights assignment, for example, of Impersonate a Client After Authentication. And it was clear that the IIS_IUSRS group had been removed from this right. (Per https://support.microsoft.com/kb/981949 it's supposed to be there.)
Honestly I'm not sure if that was the main or only user right assignment that caused this particular problem. I think it’s a likely guess but haven’t tested it specifically yet. Perhaps another user assignment right is more poignant to focus on. But I felt pretty
comfortable in taking this as a sign that group policy was overriding too much of the local policy. So IIS and/or ASP.net weren't necessarily going to work as hoped.
I'm using this as to segue to a related problem I troubleshot last week. A SharePoint Admin lost the ability to browse successfully to the Central Admin page.
Errors:
1) In the browser: An Unexpected Error Occurred
2) In the ULS Log: w3wp.exe 0x15EC SharePoint Foundation Runtime Unexpected System.Runtime.InteropServices.COMException: The handle is invalid. (Exception from HRESULT: 0x80070006 (E_HANDLE))
3) Process Monitor: “BAD IMPERSONATION”
Root Cause - Application of a Group Policy to the OU that includes the WFEs removed the IIS_IUSRS group from the local security policy on server FSAPTCTSP016 for the user right assignment of “Impersonate a client After Authentication.” The IIS_IUSRS group
needs the ability to impersonate clients after authentication because this is how the SharePoint/IIS Application Pool impersonates the SharePoint Administrator who is trying to reach Central Admin.
Recommended Solution - The local group IIS_IUSRS needs to be allowed to be listed in the Impersonate a Client after Authentication right. Please work with your Active Directory Administrator and/or Security teams to determine what needs to happen in Group Policy to make sure the Central Admin servers (or even all WFEs) have this.
I am seeing a lot of evidence of odd iis/asp.net/sharepoint related problems occurring when the IIS_IUSRs group gets pushed out of the IaCaA right. Hopefully this blog post helps a few find resolution sooner.