The Case of the Disappearing Hosts File
Hmmm...how should I phrase this?
It has been a very educational couple of weeks on my current SharePoint project.
During the rebuild of our Test environment, the SharePoint Products and Technologies Configuration Wizard failed when it was unable to find the %SystemRoot%\System32\drivers\etc\hosts file. We had encountered this error during the original installation on the SSP server in the Test environment because someone had renamed the file from hosts to hosts-old. Therefore we suspected the same problem this time around, thinking that perhaps there was some scheduled script or group policy that was disabling local host name resolution.
For those of you that may not have attempted to read the status messages as they flash by in the Configuration Wizard, step 4 changes the permissions on the hosts file to grant the WSS_ADMIN_WPG group the following permissions:
- List Folder / Read Data
- Read Attributes
- Read Extended Attributes
- Create Files / Write Data
- Create Folders / Append Data
- Write Attributes
- Write Extended Attributes
- Delete
- Read Permissions
After recreating the hosts file, I was able to successfully complete the Configuration Wizard (because the security settings on the hosts files could now be set in step 4). However, a short time later, I noticed the following in the event log:
Application Server Administration job failed for service instance
Microsoft.Office.Server.Search.Administration.SearchServiceInstance (...).
Reason: Could not find file 'D:\WINNT\system32\drivers\etc\HOSTS'.
Techinal Support Details:
System.IO.FileNotFoundException: Could not find file 'D:\WINNT\system32\drivers\etc\HOSTS'.
File name: 'D:\WINNT\system32\drivers\etc\HOSTS'
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(...)
at System.IO.FileStream..ctor(...)
at System.IO.StreamReader..ctor(...)
at System.IO.FileInfo.OpenText()
at Microsoft.Search.Administration.Security.HOSTSFile.ParseHOSTSFile(...)
at Microsoft.Search.Administration.Security.HOSTSFile.ConfigureDedicatedGathering(...)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.SynchronizeDefaultContentSource(...)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(...)
Argh! The hosts file has disappeared again!
I restored the hosts file again and set the permissions manually for the WSS_ADMIN_WPG group. However I noticed that it quickly disappeared again.
I then restored the hosts file (yet) again, but did not give the WSS_ADMIN_WPG group permission to delete the file. This resulted in the following event log entry:
Application Server Administration job failed for service instance
Microsoft.Office.Server.Search.Administration.SearchServiceInstance (...).
Reason: Access to the path 'D:\WINNT\system32\drivers\etc\HOSTS' is denied.
Techinal Support Details:
System.UnauthorizedAccessException: Access to the path 'D:\WINNT\system32\drivers\etc\HOSTS' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileInfo.Delete()
at Microsoft.Search.Administration.Security.HOSTSFile.CleanupDedicatedGathering(...)
at Microsoft.Search.Administration.Security.HOSTSFile.ConfigureDedicatedGathering(...)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.SynchronizeDefaultContentSource(...)
at Microsoft.Office.Server.Search.Administration.SearchServiceInstance.Synchronize()
at Microsoft.Office.Server.Administration.ApplicationServerJob.ProvisionLocalSharedServiceInstances(...)
Aha! So apparently the guilty party for deleting my hosts file isn't some malicious system administrator or group policy setting, but rather it is the Windows SharePoint Services Timer itself!
It turns out that this is a bug in MOSS 2007 (although I am still waiting for PSS to formally acknowledge that this is a bug). Nevertheless, I am convinced that this is a bug. Here's why:
I never recommend to customers having your service accounts be members of the local Administrators group. Quite honestly, this is simply too dangerous and presents a whole slew of risks that are beyond the scope of this posting. Since the service account that I am using is not a member of the local Administrators group, when the timer job deletes the file, it does not have permission to recreate the file. Recall that earlier I mentioned that step 4 of the Configuration Wizard only grants permissions on the hosts file itself to the WSS_ADMIN_WPG group (which the service account is a member of). Hence the disappearing hosts file.
The workaround is to grant the following permissions for the WSS_ADMIN_WPG on the %SystemRoot%\System32\drivers\etc folder:
- Traverse Folder / Execute File
- List Folder / Read Data
- Read Attributes
- Read Extended Attributes
- Create Files / Write Data
- Read Permissions
For those of you that may be wondering why SharePoint needs access to the hosts file at all, the answer is due to one of the configuration settings that you can specify for Office SharePoint Server Search.
In our case, we are using a farm configuration similar to what used to be referred in SPS 2003 as a "Medium farm" -- i.e. two front-end Web servers and one SSP server (previously referred to as the job/index server). In order to minimize the impact on users, we have the index server (i.e. the SSP) crawl itself.
In Central Administration, if you specify a dedicated front-end for crawling contant, then a timer job is created to add an entry to your hosts file to force the Web application (i.e. the host header) to resolve to that server. Unfortunately, instead of "editing" the file in-place, the developer who implemented this feature decided it would be easier to just read the hosts file, add the appropriate entries, delete the original file, and then create a new file. If your SharePoint farm account (i.e. the one that the Windows SharePoint Servers Timer runs under) is a member of the local Administrators group then there is no problem (which appears to be how this feature was tested). However, if you adhere to the "principle of least privilege" then there's definitely a problem.
Unfortunately we did not discover the bug with the disappearing hosts file in DEV because we are using a single server configuration there (and therefore did not specify a dedicated Web front-end for crawling).
I'll be writing a few more blogs posts this weekend to describe a couple of the other "gotchas" I have discovered in the last couple of weeks, but right now it is time for some pancakes!
Comments
Anonymous
November 01, 2007
Hey there! I have followed your instructions and i still get the error every minute abou the access denied to he host file... any news on this?Anonymous
January 25, 2008
Can you post the exact CACLS/XCACLS command? Obviously, CACLS %SystemRoot%System32driversetc /E /G WSS_ADMIN_WPG:C Will work, but that also grants more permissions then stated...Anonymous
February 26, 2009
The comment has been removedAnonymous
June 14, 2009
Potential Issues Installing SharePoint 2007 Service Pack 2Anonymous
August 06, 2009
Great post - thanks for the detailed explanation!Anonymous
March 22, 2010
The comment has been removedAnonymous
March 23, 2010
@James, Without knowing more details about your specific issue, it sounds like it might be the problem described in KB 896861, and also mentioned in the following post: http://blogs.msdn.com/jjameson/archive/2009/02/10/issues-with-running-moss-2007-on-windows-server-2008.aspx Hope that helps.Anonymous
April 15, 2010
This did not resolve my issue either. Is a reboort or stop and start of services required?Anonymous
April 15, 2010
Nope...I've never had to reboot or restart services. I recommend you call Microsoft Support. It's a bug -- you shouldn't be charged for the support incident.Anonymous
April 15, 2010
I should clarify... If your problem is caused by a missing hosts file, then once you fix the permissions you shoudn't need to reboot. (The next time the SharePoint timer job runs, it will successfully write out the updated hosts file.) If, on the other hand, your problem is caused by the "loopback" issues described in my other post, then yes, you definitely need to reboot after applying the fix described in the KB article.Anonymous
June 06, 2010
Hi, placing the proper rights on the folder as described .... It didn´t solve my problem either ..... but then I renamed my own replaced hosts file. The timer job then created a HOSTS file on it´s own, so it did solve my problem actually ..... The only proble was that the timer job had to create the file itself ...Anonymous
June 06, 2010
Hi, placing the proper rights on the folder as described .... It didn´t solve my problem either ..... but then I renamed my own replaced hosts file. The timer job then created a HOSTS file on it´s own, so it did solve my problem actually ..... The only proble was that the timer job had to create the file itself ...Anonymous
April 06, 2011
I just encountered the disappearing hosts file issue. However my issue was not around permissions. The behavior I noticed is that the hosts file was moved and renamed to the ULS Logs folder by the Farm account. Did anyone observe this behavior too ? thanks a lot for your post. mark