Udostępnij za pośrednictwem

Store Background Processes Part I - IS Maintenance

  • Artykuł

Overview

IS Maintenance is the term applied to a series of operations performed by the Information Store to ensure logical consistency in the databases. Generally speaking the operations ensure items no longer needed by the system are removed from the database. If a custom schedule has not been defined for the database in Exchange System Manager, a default scheduled period of 12-5AM local time will be used.

Overall, there are 10 tasks performed by the Information Store during this period:

  1. Purge Indices (public and mailbox stores)

  2. Tombstone Maintenance (public and mailbox stores)

  3. Dumpster Cleanup (public and mailbox stores)

  4. Public Folder Expiry (public stores)

  5. Age Folder Tombstone (public stores)

  6. Folder Conflict Aging (public stores)

  7. Update Server Versions (public stores)

  8. Cleanup Secure Folders (public stores)

  9. Cleanup Deleted Mailboxes (mailbox stores)

  10. Check Messages Table (public and mailbox stores)

If the maintenance window ends before one of the ten tasks above is completed, the last one running is recorded. The last task will be allowed to run until completion, which could exceed the maintenance window. During the next maintenance window the information store determines the last task in progress and restarts the process, essentially picking up where it left off on the next task.

The progress of the tasks above can be monitored by adjusting the Event Logging Level to minimum for the Exchange Information Store Mailbox/Public. When set to minimum, the following events are logged:

Event Type: Informational

Event Source: MSExchangeIS Public / Private

Event ID: 1208

User: N/A

Description:

Starting the IS Maintenance tasks.

%n%nFor more information, click https://www.microsoft.com/contentredirect.asp.

Event Type: Informational

Event Source: MSExchangeIS Public / Private

Event ID: 1210

User: N/A

Description:

The IS Maintenance task <subtask name> completed.

%n%nFor more information, click https://www.microsoft.com/contentredirect.asp.

Event Type: Informational

Event Source: MSExchangeIS Public / Private

Event ID: 1209

User: N/A

Description:

The IS Maintenance tasks completed.

%n%nFor more information, click https://www.microsoft.com/contentredirect.asp.

I want to delineate the 11th task performed, Online Defragmentation, which is not performed by the Information Store but by the underlying database engine (ESE). At the completion of the above list of tasks, the Store process initiates Online Defragmentation by calling into (ESE) to inform it to start the defragmentation process. It is only after at least 1 of the 10 tasks above have been successfully completed will this process start.

The progress of online defragmentation can be monitored by reviewing the application log without adjusting the Event Logging levels.

When the Information Store initiates ESE online defragmentation for a database, the following is logged:

Event Type: Informational

Event Source: ESE

Event ID: 700

User: N/A

Description:

%1 (%2) %3Online defragmentation is beginning a full pass on database '%4'.

%n%nFor more information, click https://www.microsoft.com/contentredirect.asp.

If a full pass of online defragmentation completes within the maintenance window the following is logged indicating it has completed.

Event Type: Informational

Event Source: ESE

Event ID: 701

User: N/A

Description:

%1 (%2) %3Online defragmentation has completed a full pass on database '%4'.

%n%nFor more information, click https://www.microsoft.com/contentredirect.asp.

 

If a pass of the online defragmentation was interrupted due to window expired or backup initiated, the following event is logged:

Event Type: Informational

Event Source: ESE

Event ID: 704

User: N/A

Description:

%1 (%2) %3Online defragmentation of database '%4' was interrupted and terminated. The next time online defragmentation is started on this database, it will resume from the point of interruption.

%n%nFor more information, click https://www.microsoft.com/contentredirect.asp.

If the previous online defragmentation cycle was previously interrupted before it completed a full pass of the database the following is logged instead of the event above:

Event Type: Informational

Event Source: ESE

Event ID: 702

User: N/A

Description:

%1 (%2) %3Online defragmentation is resuming its pass on database '%4'.

%n%nFor more information, click https://www.microsoft.com/contentredirect.asp.

If a resumed pass of the database completes within the maintenance window the following is logged:

Event Type: Informational

Event Source: ESE

Event ID: 703

User: N/A

Description:

%1 (%2) %3Online defragmentation has completed the resumed pass on database '%4'.%n%nFor more information, click https://www.microsoft.com/contentredirect.asp.

Things to keep in mind…

An important note to make about IS Maintenance and online defragmentation is the large number of changes made to the database. To maintain recoverability of the database, each of the changes made by these tasks are also transacted in the transaction logs. Depending upon the number of changes made in the database during these tasks, a large number of log files can be generated during the maintenance windows.

Backups schedules should also be monitored so they do not overlap with the IS Maintenance schedule. Backup will not preempt the 10 initial tasks performed by the Information Store, therefore they will run simultaneously. However, when initiating a backup of a database, the online defragmentation process is suspended until the backup is completed.

Below is a list of every action during Online Maintenance and the possible configuration operations to control their behavior.

Purge Indices (public and mailbox stores)

One of the benefits of the Extensible Storage Engine (ESE or aka JET) is its ability to dynamically create indexes when information is resorted or requested in a different sort order. Unchecked, the Information Store database could end up with thousands of indexes on a table.

The Information Store handles this situation by keeping track of indexes by assigning them an expiry time. Information about the index is added to an internal table referred to as the indexing aging table. During online maintenance the index aging table is walked and each entry’s value of ptagIndexDeleteTime is compared to the current time. If the index has expired, the index is removed. This process runs every 24 hours during the Information Store maintenance cycle and removes entries older than 8 days for Exchange 5.5 and 40 days for Exchange 2000/2003 by default.

Default Inverval:               Every 24 hours

Default Criteria:                 Remove Indexes older than 8 days on Exchange 5.5

and older than 40 days on Exchange 2000/2003.

Configurable Settings:

DC=<Domain>

CN=Configuration

              CN=Services

                     CN=Microsoft Exchange

                           CN=<ExchOrgName>

                                  CN=Administrative Groups

                                         CN=<site>

                                                CN=Servers

                                                       CN=<servername>

                                                              CN=InformationStore

                                                                     CN=<Storage Group>

                                                                           CN=<Database>

Object Class: Database Object

Attribute Name: msExchAgingKeepTime

Description:                The amount of time in milliseconds seconds to keep unused indexes and views in the database. This value is applies to Exchange 2000/2003 and supersedes the registry values below.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem

Value Name:                 Aging Keep Time

Radix: REG_DWORD

Description:                The amount of time in milliseconds seconds to keep unused indexes and views in the database. This value is honored by both Exchange 5.5 and 2000/2003.

Value Name: Aging Clean Interval

Radix: REG_DWORD

Description: The interval (in milliseconds) at which the store attempts to purge unused indices. This value is honored by both Exchange 5.5 and 2000/2003.

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

- or –

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPrivate

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

-or-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Private-<Public Folder GUID>

Value Name: Reset Views

Radix: REG_DWORD

Description: Setting this value to 1 will cause the Store to remove all indexes regardless of age on the next cycle of IS maintenance for the corresponding database. The value is then set back to 0. By setting this value, clients may experience initial delays rebuilding previously cached views.

Tombstone Maintenance (public and mailbox stores)

Each folder maintains a list of deleted messages for a folder in ptagMidsetDeleted on the folder object itself. When a message is deleted from a folder, an entry is made in ptagMidsetTombstones. If this folder is replicated, this gives an indication of the messages delete operations that need to be propagated. When the propagation occurs, the entries are moved from ptagMidsetTombstones and merged into ptagMidsetDeleted. If a folder is not replicated, it is necessary to clean up this situation through the Information Store maintenance cycle. This process runs every 24 hours and is not configurable.

Dumpster Cleanup (public and mailbox stores)

When a message is deleted from a client a simple flag is set to indicate (ptagMsgDeleted) the message is deleted. By default a client will not display messages that have this flag set. In order to view these messages you use the dumpster functionality in the client. If the dumpster is not turned on, messages that have been deleted will be removed from the folder when this process runs during the maintenance cycle. If dumpster functionality has been enabled, only messages that have been deleted and have exceeded the retention period will be removed during this maintenance routine.

This process also checks for folders that have exceeded the retention period and will remove them as well.

Configurable Settings:

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

- or –

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPrivate

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

-or-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Private-<Public Folder GUID>

Value Name: Deletion Thread Period

Radix: REG_DWORD

Description: The interval (in seconds) at which the store attempts to purge unused indices. This value is honored by both Exchange 5.5 and 2000/2003.

Public Folder Expiry (public stores)

This is a maintenance process of traversing through all the messages in Public Folders and removing the ones that have exceeded the expiry time.

Configurable Settings:

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

Value Name: Replication Expiry

Radix: REG_DWORD

Description: The interval (in milliseconds) at which the store removes items that have exceeded the retention period.

Age Folder Tombstone (public stores)

This maintenance process removes deleted public folders that have exceeded the tombstone lifetime (180days default). When a Public Folder is deleted, it is tombstoned for replication purposes. This allows for items that are deleted to be tracked during the replication process. This removes a maximum of 500 folders per a 24 hour period.

Configurable Settings:

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

Value Name: Replication Folder Tombstone Age Limit

Radix: REG_DWORD

Description: The interval (in days) to keep tombstones.

Folder Conflict Aging (public stores)

When two users have edited and modified the same message in a public folder, the resulting saved messages are defined as being in conflict. This can happen when a simultaneous save occurs on the same public folder server, or if the message is edited on two servers that contain replicas of the folder. If no action is taken by the owners of the Public Folder within Conflict Age Limit, this process takes the appropriate action based upon the property PR_RESOLVE_METHOD on the folder itself. The resolution method is defined in the Exchange System Manager on the Public Folder object.

Configurable Settings:

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

Value Name: Replication Folder Conflict Age Limit

Radix: REG_DWORD

Description: The interval (in days) items in conflict are allowed to exist.

Update Server Versions (public stores)

This process simply updates the version information as necessary for any public folder databases containing a replica of the system configuration folder. No configurable settings are available.

Cleanup Secure Folders (public stores)

This process is only valid for Exchange 5.5. This process attempts to age out folders that have been secured in other sites and the site is no longer present in the directory. Each folder is examined and for each secure folder where it’s home site is not visible in the directory a timer is set to remove the folder. The default is 7 days.

Configurable Settings:

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

Value Name: Secure Folder Aging Task Frequency

Radix: REG_DWORD

Description: The interval (in days)

Site Folder Check (public stores)

Every 1 hour a public database will check to ensure no duplicate site folders exists within an Administrative Group and removes them. No configurable settings are available for this process.

Cleanup Deleted Mailboxes (mailbox stores)

This process cycles through mailboxes that currently do not have Directory Service objects. After a default of 30 days, the mailboxes are deleted. This process runs every 24 hours and no interval adjustments are available.

Configurable Settings:

DC=<Domain>

       CN=Configuration

              CN=Services

                     CN=Microsoft Exchange

                           CN=<ExchOrgName>

                                  CN=Administrative Groups

                                         CN=<site>

                                                CN=Servers

                                                       CN=<servername>

                                                              CN=InformationStore

                                                                     CN=<Storage Group>

                                                                           CN=<Database>

Object Class: Database Object

Attribute Name: msExchMailboxRententionPeriod

Description:                The amount of time in seconds to keep retain mailboxes without a corresponding Directory Object.

Check Messages Table

This process traverses the message table for a given database and looks for messages that currently do have a reference count of 0 indicating no folder currently has a reference to the message. Messages encountered are deleted. A maximum of 50000 messages will be deleted in a given cycle. This process runs every 24 hours and no interval adjustment is available.

Online Defragmentation

If at least one of the maintenance task was able to complete within the maintenance schedule, the Information Store requests that ESE begin an online defragmentation cycle and complete a single full pass of the database. This process will run for at least 15 minutes at the end of the maintenance window.

The intention for online defragmentation is to free up pages in the database by compacting records onto the fewest number of pages possible, thus reducing the amount of I/O necessary. The ESE database engine does this by walking the database metadata (information in the database that describes tables in the database) and for each table, visit each page in the table and attempt to move records onto logically order pages. If you have an understanding of BTrees, we start at the furthest page to the right and begin compressing records to the left most page. This does not necessarily mean the pages are in order, but the movement is from a logical perspective.

It is possible this process does not complete before the time indicated by the Information Store. In such a case the ESE engine stores the last table successfully processed. At the next Online Defragmentation request, ESE picks up on the next table in the metadata will complete the work until all tables have been processed or the maintenance window ends. If Online Defragmentation is able to process all tables before the maintenance window ends, it will restart at the first table listed in the metadata and continue processing until all tables are processed or the maintenance window ends.

It is import to point out however, if IS maintenance is set to always and one of the above tasks completes (note some only run once every 24 hours), then Online Defragmentation could be invoked.

Configurable Settings:

For Exchange 5.5:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\ParametersPublic

For Exchange 2000/2003:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Public-<Public Folder GUID>

-or-

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS

\<ServerName>\Private-<Public Folder GUID>

Value Name: OLD Minimum RunTime

Radix: REG_DWORD

Description: Amount of time to run after completing at least 1 of IS Maintenance tasks (in minutes)

                           Default value of 15 minutes

Value Name: OLD Completion Time

Radix: REG_DWORD

Description: The amount of time (in seconds) beyond the maintenance window Online Defragmentation will be allowed to run.
Default value of 1 hour (3600sec)

EDITS:
9/9/04 - Adjusted units for Aging Keep Time from milliseconds to seconds.

9/28/04 - Clarified Online Defragmentation process

9/30/05 - Changed " Deletion Thread Period" milliseconds to seconds

Comments