Exchange 2013: High Availability, FAST Search, and the Windows Registry

One of the things not mentioned about High Availability (HA) and Database Availability Groups (DAGs) is that the automatic reseed feature is also used to maintain Exchange FAST Search indexes, independently, by HA.

HA utilizes a registry key to keep the Content Index (CI) states in a central locale for referencing. It queries this key because determining if the service is running on each node is approximately 30% of the cost of the 'Get-MailboxDatabaseCopyStatus' command. The registry key, itself, is located at: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\Search\IndexStatus

You can query this registry key using the following syntax in PowerShell or Exchange Management Shell:

Get-ItemProperty HKLM:\SOFTWARE\Microsoft\ExchangeServer\v15\Search\IndexStatus

Assuming your mailbox server is not a member of a DAG, when you run the command you'll get outputs that look something like the following:

{79853a57-6c9e-46a2-a5e3-70db036754f1} : 1,1,4294967297,2013-07-16 00:33:17Z,0,
{115d21d9-8b88-4574-b71e-5a7142daaf78} : 1,1,4294967297,2013-07-16 00:33:16Z,0,
{aab61984-7df3-412f-9c0b-f44bf0eaadbc} : 1,1,4294967297,2013-07-16 00:33:16Z,0,

Assuming your mailbox server is a member of a DAG, when you run the command you’ll get outputs that look something like the following:

{b03445e5-58c1-4f46-9d3b-3db2cb3ee3e3} : 1,1,2013-07-16 14:27:56Z,0,
{94689a79-5207-42a0-8ebc-50afe9fbfb52} : 4,7,2013-07-16 14:39:05Z,0,SPECIAL:E15-DAGMEMBER-1/3875/94689A79-5207-42A0-8EBC-50AFE9FBFB5212/Single
{1bfb37b0-9a9c-4821-9d03-cf0c7f8f2bba} : 4,7,2013-07-16 14:39:04Z,0,SPECIAL:E15-DAGMEMBER-5/3875/1BFB37B0-9A9C-4821-9D03-CF0C7F8F2BBA12/Single
{c77b6a95-719b-477c-be95-504f81b5376d} : 1,1,2013-07-16 14:14:02Z,0,
{1194860a-35c9-452e-8e87-b040eea1909c} : 4,7,2013-07-16 14:39:04Z,0,SPECIAL:E15-DAGMEMBER-2/3875/1194860A-35C9-452E-8E87-B040EEA1909C12/Single

There’s five points of information that make this relevant to issues with FAST Seach: 

1.)    The initial {GUID} is the GUID of the database.

1. There will be 1 entry per database on the DAG node.

2.)    ‘SPECIAL’ means that the CI is reseeding

1. The source server, port, source database, and cell instance trail this value

3.)    The double numeric values (1,1; 4,7; 8,8; etc.) signify the state[s] of the CI

4.)    'Single' references which type of cell we are dealing with.

5.)    The two digit number, after the second GUID, signifies the schema version of the index (in this case, the schema version is 12). You will not be able to find the database in question, if you use the second GUID and leave the schema version appended to it.

 

The first number in the series is the Index Status. The second number in the series is the error code. The codes break-down as follows:

Index Status:

Unknown = 0
Healthy = 1
Crawling =2
Failed = 3
Seeding = 4
Failed And Suspended = 5
Suspended = 6
Disabled = 7
Auto-Suspended = 8
Healthy And Upgrading = 9

Error Code:

1 Success
2 Internal Error         
3 Crawling Database
4 DatabaseOffline
5 MapiNetworkError
6 Catalog Corruption
7 Seeding Catalog         
8 Catalog Suspended         
9 Catalog Reseed
10 Index Not Enabled         
11 Catalog Excluded
12 Activation Preference skipped         
13  Lag Copy skipped         
14 Recovery Database Skipped         
15 Fast Error         
16 Service Not Running
17 Index Timestamp too Old    

When a CI is reseeding, the status will – obviously – be ‘Seeding’ when you run ‘Get-MailboxDatabaseCopyStatus’. In order to see where the seeding source is, find the ‘ContentIndexSeedingSource’ value for that specific copy via 'Get-MailboxDatabaseCopyStatus' or you can query the registry key. Keep in mind, though, that the source port and cell are not exposed via ‘Get-MailboxDatabaseCopyStatus’ and are only found in the registry.

 

Credits:

• Scott Oseychik, Principal Escalation Engineer, Customer Service and Support
• Microsoft Exchange FAST Team
• Microsoft Exchange Sustained Engineering Team

Comments

• Anonymous
  January 01, 2003
  Apologies for the late reply, Mike. It does look like some changes were made, as my values show the same for SP1 (what version is your DAG member[s] on?).
  
  {79853a57-6c9e-46a2-a5e3-70db036754f1} : 5,9,4294967297,2013-10-25 10:04:25Z,0,
  {115d21d9-8b88-4574-b71e-5a7142daaf78} : 5,9,4294967297,2013-10-25 10:04:35Z,0,
  {aab61984-7df3-412f-9c0b-f44bf0eaadbc} : 5,9,4294967297,2013-10-25 10:04:37Z,0,
  {660a41a1-7196-4bd9-bb12-f76ee86539e7} : 6,8,8589934594,2014-04-12 23:16:53Z,0,
  {135d5536-d180-4198-9ba8-574b53df8206} : 1,1,4294967297,2014-04-12 09:54:01Z,0,
  {ff497057-40b2-4aa8-a65d-76f57d7a0181} : 1,1,4294967297,2014-04-12 09:54:08Z,0,
  {7f1ea4c7-0b54-4df2-9c35-fe2c605fc71e} : 3,4,4294967297,2014-01-11 03:45:53Z,0,
  {17a1a01b-3cc6-4bc2-88ff-7ce387aa867b} : 3,4,42949672970,2014-01-30 04:37:42Z,0,
  
  I'll follow-up and see if I can get context as to what changed, so I can update this.
• Anonymous
  March 06, 2014
  Scott, all 240 Exchange 2013 Databases appear as non DAG databases per your document. Has anything changed since RTM?
  
  Value for DAG DB: 1,1,4294967297,2014-03-06 16:27:30Z,0,
• Anonymous
  March 06, 2014
  My mistake, I mean John.
• Anonymous
  April 14, 2014
  These articles describe how to troubleshoot Fast Search issues with Exchange Server 2013. When you think