SharePoint 2007: Configure a Dedicated Front-End Web Server for Crawling By Editing the Hosts File

Article
1/17/2024

Introduction

Before you perform this procedure, confirm that you have read the topic Configure a dedicated front-end Web server for crawling (Office SharePoint Server 2007).

Use the following procedures in the order listed to configure a dedicated front-end Web server for crawling by editing the Hosts file directly.

Check Network Load Balancing Settings

Because crawling content typically places the front-end Web server under heavy load, we recommend that you remove the dedicated front-end Web server from the network load balancing rotation. Otherwise, user requests that the network load balancer directs to the dedicated front-end Web server for crawling will be subjected to inconsistent performance. For more information about network load balancing, see Remove a host from a Network Load Balancing cluster (http://go.microsoft.com/fwlink/?LinkID=91030&clcid=0x409).

Start the Windows SharePoint Services Web Application

You must ensure that the Windows SharePoint Services Web Application service is running on the front-end Web server that you want to use for crawling content. You must be a farm administrator to complete this procedure.

In Central Administration, on the Operations tab, in the Topology and Services section, click Servers in farm.
On the Servers in Farm page, in the Server column, click the name of the server you want to configure as your dedicated front-end Web server for crawling.
On the Services on Server page, in the Start services in the table below section, find the Windows SharePoint Services Web Application row. Next verify that the Status column displays Started. If this service is started, no further action is needed. Otherwise, click Start.

Specify the Server to use for Crawling

Before you can configure the Hosts file, you must collect the following information:

The base or host header name of each Web application in your farm that you will crawl.
IP address of the server you want to use as the dedicated front-end Web server for crawling.

Record the base or Host Header names

In Central Administration, on the Application Management tab, in the SharePoint Web Application Management section, click Web application list.
On the Web Application List page, in the URL column, note the base or host header names of the Web applications that you want to crawl. For example, if the URL is http://contoso, the base or host header name is contoso.

Record the IP address of the Server to be used for Crawling

If you configure the index server as the dedicated front-end Web server for crawling, you can skip to Verify the Hosts file. Otherwise, note the IP address of the server that you want to configure as the dedicated front-end Web server for crawling. If you know the NetBIOS name of the server, you can use the ping command to determine the IP address.

You must be a member of the administrators group on the index server to do the following procedure

Verify the Hosts file

In Windows Explorer, navigate to the <systemdrive>:\winnt\system32\drivers\etc folder.
Open the Hosts file in Notepad.
On the line below the entry 127.0.0.1 localhost, add the entries for the Web applications that you want to crawl.
On the File menu, click Save As.
In the Save as type list, select All Files.
Click Save and then close Notepad.

Hosts file Examples

The following examples show how you might configure the Hosts file for different scenarios. These scenarios assume the following:

You have removed the server that you are specifying as the dedicated front-end Web server for crawling from network load balancing rotation, if applicable. For more information about network load balancing, see Remove a host from a Network Load Balancing cluster
You have started the Windows SharePoint Services Web Application service on the server that you are specifying as the dedicated front-end Web server for crawling, as described earlier in this article.

Scenario 1 - Configure an Index Server for Crawling

Your server farm has three Web applications that you crawl. The application names are Contoso, Marketing, and Support. Because you are using the index server as the front-end Web server for crawling, you can use the loopback IP address (127.0.0.1) to specify that the index server as the front-end Web server for crawling.

You add the following entries to your Hosts file below the entry 127.0.0.1 localhost:

127.0.0.1 Contoso
127.0.0.1 Marketing
127.0.0.1 Support

Scenario 2 - Configure a Dedicated Crawling Server

Your server farm has four Web applications that you want to crawl. The application names are Contoso, Marketing, Sales, and Support. Because you are not using the index server as the Web front-end for crawling, you must add the actual IP address of the server you want to specify as the dedicated Web front-end for crawling. In this example, the IP address of the dedicated front-end Web server for crawling is 10.10.10.4.