Share via

Search and Replace Retention tag on Microsoft Exchange 2010 (MRM)

  • Article

Messaging Records Management
development on Retention Policy Tag

 

          
Introduction

Microsoft introduces the messaging record management (MRM). In Exchange 2010, which helps the organizations to reduce legal risks associated with e-mail and other communications.

MRM makes it easier to keep messages needed to comply with company policy, government regulations, or legal needs, and to remove content that has no legal or business value.

To learn more about MRM and how to set it up, please visit https://technet.microsoft.com/en-us/library/dd297955.aspx

In this article, will go over what are retention policy tags? How it is being internally stored in MAPI objects? How programmatically change the retention policy tags in each item of the exchange.

              
What are retention policy tags?

MRM in Exchange 2010 is accomplished by using retention tags and retention policies. Before discussing the details about each of these retention features, it's important to learn how the features are used in the overall Exchange 2010 MRM strategy. This strategy is based on:

  • Assigning retention policy tags (RPTs) to default folders, such as the Inbox 
  • Applying a default policy tag (DPT) to mailboxes to manage the retention of all untagged items 
  • Allowing the user to assign personal tags to custom folders and individual items 
  • Separating MRM functionality from users' Inbox management and filing habits.
    • Users
      aren't required to file messages in managed folders based on retention
      requirements
    • Individual
      messages can have a different retention tag than the one applied to the folder
      in which they're located

Figure 1 illustrates
the tasks involved in implementing this strategy

 
  
 
 
Figure 1

Ref:  https://technet.microsoft.com/en-us/library/dd297955.aspx


Retention Tags Types (Reference: https://technet.microsoft.com/en-us/library/dd297955.aspx)

As illustrated in Figure 1, retention tags are used to apply retention settings to folders and individual items such as messages, notes, and contacts. These settings specify what policy a message should have, how long a message remains in a mailbox and the action to be taken when the message reaches the specified retention age. When a message reaches its retention age, it's moved to the personal archive, deleted, or flagged for user attention.

Unlike managed folders (the MRM feature introduced in Exchange 2007), retention tags allow users to tag mailbox folders and individual items for retention. Users no longer have to file items in managed folders based on message retention requirements.

Retention tag configuration objects are stored in Active Directory (AD) in the Retention Policy Tag Container, which is located under the organization container. There are three types of retention tags:

  • Retention Policy Tags
  • Default Policy Tags
  • Personal Tags

Retention Policy Tags (RPTs)

RPTs apply retention settings to default folders such as Inbox, Deleted Items, and Sent Items. Mailbox items in a default folder that have an RPT applied inherit the folder's tag. Although users can't apply a different tag to a default folder, they can apply a different tag to the items in a default folder.

You can create RPTs for the following default folders:

Calendar (requires Exchange 2010 SP1)

Notes (requires Exchange 2010 SP1)

Deleted Items

Outbox

Drafts

Sent Items

Inbox

RSS Feeds

Journal

Sync Issues

Junk E-mail

Conversation History

 

      Important:

  

  

You can't include more than one RPT for the same   default folder type in one retention policy. For example, if a retention   policy has an Inbox tag, you can't add another RPT of type Inbox to that   retention policy.

  

 

In Exchange 2010 RTM, RPTs are not supported for the Calendar and Notes default folders. In Exchange 2010 RTM and SP1, retention tags are not applied to Contacts. 

Default Policy Tags (DPTs)

DPTs apply retention settings to untagged mailbox items. Untagged items are mailbox items that do not already have a retention tag applied, either by inheritance from the folder in which they are located or by the user. A retention policy cannot contain more than one DPT with the same action.

Personal Tags

Personal tags are available to Outlook 2010 and Outlook Web App users as part of their retention policy. Users can apply personal tags to folders they create or to individual items, even if those items already have a different tag applied.

 
Retention Policy Tag Attributes

MRM depends on the Active Directory (AD) directory to store configuration object settings used for driving Managed Folder assistant provisioning and retention actions. These configuration settings are retrieved and set using Exchange management tasks from EMS, EMC and ECP

Retention tag configuration objects are of objectClass msExchELCFolder and are located in the Retention Policy Tag Container directly under the Organization container. msExchgELCFlags, and msExchELCFolderTypes  are the attributes used by MRM.  For each retention tag there is a Content
Settings configuration object of objectClass msExchELCContentSettings that is a direct child object of the retention tag configuration object.  Table 1 describes the attributes used by MRM.

Table 1: Content Setting Configuration Attributes

  

  

Attribute

  

msExchELCAutoCopyAddressLink

msExchELCExpiryAction

msExchELCExpiryAgeLimit

msExchELCFlags

msExchELCLabel

msExchELCMessageClass

 

Retention Policy Attributes

Retention policy configuration objects are of objectClass msExchRecipientTemplate and are located in the Retention Policies Container directly under the Organization container. Table 2 describes the attributes used by MRM.

Table 3: Retention Policy Configuration Attributes

  

  

Attribute

  

msExchELCFolderLink

msExchMailboxTemplateBL

msExchMinAdminVersion

msExchRecipientTemplateFlags

 

Mailbox Objects

User’s mailbox contains Information specific to MRM  operation. This information is used by the Managed Folder Assistant for processing retention and archival actions, and by mailbox clients for displaying and managing relevant retention and archival elements in the user interface.

Configuration Message

The MRM configuration information for a mailbox to which a retention policy has been applied is stored in a hidden Folder Associated Item (FAI) message of type IPM.Configuration.MRM, located in the Associated Contents table of the Inbox folder.  This message in the mailbox is not visible in Outlook or OWA to the end user. MRM uses the FAI item to communicate the Retention Policy to the mailbox client, and for processing retention tags that have been self-provisioned by the user.

The configuration information is stored as XML formatted data in property PR_ROAMING_XMLSTREAM (0x7C080102). This information can be examined using the MAPI Editor tool (MFCMAPI).

 

      Important:

  

  

Once a   retention tag is applied to a mailbox and the information for that tag is   written to the FAI message, the information remains even if the tag is   un-provisioned by the user or the tag is no longer applied by policy. When   this occurs the IsVisible attribute is set to False so the user no longer   sees the tag from the client.

  

However, the   Managed Folder Assistant continues to process any messages that are tagged   with the hidden retention tag as long as the configuration object for the   tag is still available in AD. When the tag is removed from AD, the   information for the tag is also removed from all FAI messages by the Managed   Folder Assistant.

  

 

Updating the Configuration Message

The configuration information is updated by one of three
ways:

  • Managed Folder Assistant – The Managed Folder Assistant creates the configuration message when a retention policy is first applied to the mailbox, and updates the message anytime changes are made to the retention tags or retention policy applied to the mailbox.
  • Set-RetentionPolicyTag – This cmdlet makes it possible for the administrator to assign additional retention tags that are not already included in the retention policy applied to the mailbox. When the administrator adds or removes retention tags via this command, the configuration message is updated.
  • Self-Provisioning of Retention Policies via ECP – When a user adds or removes retention tags via the ECP, the configuration message is updated. The administrator can retrieve a list of all retention policies applied to a mailbox by policy assignment, administrative assignment or self-provisioning by using the Get-RetentionPolicyTag cmdlet with the Mailbox parameter. The command reads directly from the configuration message to retrieve the current settings. Because of this the mailbox must be available for this operation to succeed.

Folder Properties

MRM stores retention settings as MAPI property values on mailbox folders. Table 4 describes these property values.

Table 4: Retention Setting Properties on Mailbox Folders

  

  

Property Name

  

  

  

propTagID

  

  

  

Type

  

  

  

Stamping Details

  

StartDateEtc

PR_START_DATE_ETC

0x30190102

GUID

The GUID of the retention tag applied to the folder. Stamped on all  folders. Outlook changes this property when the user explicitly tags a  folder. Outlook updates all the subfolders with this tag’s GUID as well.

RetentionPeriod

PR_RETENTION_PERIOD

0x301A0003

Number

Of Days

The retention period for keeping the item (if it’s a value like 0 or -1  it means never expire). Stamped on all folders.  Outlook changes this property when the user  explicitly tags a folder. Outlook updates all the subfolders with the  retention period as well.

RetentionFlags

PR_RETENTION_FLAGS

0x301D0003

Number

The property representing if the recipient tag is inherited from a  parent folder. Outlook stamps the property in offline/cached mode, Exchange  stamps for online mode. If the user tags a folder, RetentionFlags is 0 for  each sub-folder and non-zero for the folder that is explicitly tagged. If the  least significant bit is 0 or if the property is not present, then the tag is  implicit.    

See RetentionFlags table for details.

 

 
 
 

Item Properties

MRM stores retention settings as MAPI property values on mailbox items. Table 5 describes these property values.

Table 5: Retention Setting Properties on Items

  

  

Property Name

  

  

  

propTagID

  

  

  

Type

  

  

  

Stamping Details

  

StartDateEtc

PR_START_DATE_ETC

0x301B0102

DateTime

This is a composite prop containing the Start Time and Default  Retention Period. The first 4 bytes is the default retention period and the  next 8 bytes is the date stamped on every item. Exchange Event-Based MRM  Assistant sets this property.  Outlook changes if necessary (Calendar and Task Item) Add the policy length  to the start time to calculate expiry date.

RetentionPeriod

PR_RETENTION_PERIOD

0x301A0003

Number

(In Days)

The time period for keeping the item (if it’s a value like 0 or -1 it  means never expire). Stamped only when item is explicitly tagged. Outlook  changes the value when explicitly tagged.

DefaultRetentionPeriod

<same as Start Time>

Number

The length to retain an item if it is under the default policy (varies  based on message class). Stamped on every item by Exchange  Event-Based/Time-Based Assistant.  Never stamped by Outlook.

RetentionDate

PR_RETENTION_DATE

0x301C0040

DateTime

DateTime for item expiration. This is a calculated prop when cached or  offline and a property stamped by Exchange when online. Stamped on every  item. Exchange stamps when online, Outlook calculates when cached or offline.

PolicyTag

PR_POLICY_TAG

0x30190102

GUID

The retention policy an item is under (either implicit or explicit).  Stamped on every item. Outlook calculates when cached or offline. Exchange  stamps when online.

RetentionFlags

PR_RETENTION_FLAGS

0x301D0003

Number

The property representing if the recipient tag is inherited from a  parent folder. Outlook stamps the property in offline/cached mode, Exchange  stamps for online mode. If the user tags an item, RetentionFlags is non-zero  for the item that is explicitly tagged. If the least significant bit is 0 or  if the property is not present, then the tag is implicit.

See RetentionFlags table for details.

ArchiveTag

PR_ARCHIVE_TAG

0x30180102

GUID

The archive policy an item is under (either implicit or explicit). Stamped  on every item. Outlook calculates when cached or offline. Exchange stamps  when online.

ArchivePeriod

PR_ARCHIVE_PERIOD

0x301E0003

Number

(In Days)

The time period for archiving the item (if it’s a value like 0 or -1 it  means never expire). Stamped only when item is explicitly tagged. Outlook  changes the value when explicitly tagged.

ArchiveDate

PR_ARCHIVE_DATE

0x301F0040

DateTime

DateTime for item archival. This is a calculated prop when cached or  offline and a property stamped by Exchange when online. Stamped on every  item. Exchange stamps when online, Outlook calculates when cached or offline.

 

Table 5 describes the values used for the RetentionFlags property on mailbox folders and items.

Table 5: RetentionFlags Bitmask Values

  

  

RetentionFlags     

  

  

  

Bitmask      

  

  

  

Hex    

  

  

  

Dec

  

None

00000000 00000000

0x00

0

ExplicitTag

00000000 00000001

0x01

1

UserOverride

00000000 00000010

0x02

2

Autotag

00000000 00000100

0x04

4

PersonalTag

00000000 00001000

0x08

8

AllRetentionFlags

00000000 00001111

0x0F

15

ExplictArchiveTag

00000000 00010000

0x10

16

KeepInPlace

00000000 00100000

0x20

32

AllArchiveFlags

00000000 00110000

0x30

48

SystemData

00000000 01000000

0x40

64

NeedsRescan

00000000 10000000

0x80

128

PendingRescan

00000001 00000000

0x100

256

 

    
Exchange Web Service

Microsoft Exchange Server 2010 provides Exchange Web Services as an extensibility point for clients that connect to the Exchange server and consume information about user availability, and the manipulation of items that are located in the Exchange data store. 

To learn more about Exchange web service please click here https://msdn.microsoft.com/en-us/library/bb204119.aspx

 
Programmatically change the retention policy tag on the Exchange

 Business Scenario

For my current customer, we had a scenario, using PowerShell the admin needs to

  • Search for all items which have a particular retention tag assigned AND belong to a specified policy
  • Specify the new retention tag in that command and have it assigned to the found mail items

We don’t have out-of-box functionality in Exchange 2010 SP1 for the above requirement. Hence, I have to do custom coding.


Solution

The custom script created as a framework. Using this script admin can search and replace the retention policy tag. Also, just by modifying the script, admin can search on how many users having certain policy tags, get the policy tag for the specific user, get the policy tag for the specific folder, and  get the expiration date.

Below is the conceptual architecture, and code.

Pre-requisite

  • Only admin can run this utility as it needs impersonation (to search on other mailboxes)
  • PowerShell 2.0
  • Exchange 2010 SP2 Web Service SDK
  • Replaceable tag must be part of the same retention policy.

Conceptual Architecture

 

    Figure 2

 As shown in the Figure 2, PowerShell will read the users from the UserAccounts.txt file.  Admin will be providing the description of the retention policy tag, but internally MAPI stores only the GUID of the tag. Hence, there will be a look-up function to match the tag description with the GUID, for both search and replace tags. Then
the Exchange Web Service (EWS) will be invoked to impersonate each user’s mailbox and search all the folders.  Upon search, if match found the GUID will be replaced with the replace tag GUID provided by the admin. Report of the operations will be written in the SharePoint.

 


Code

 #This Script is to replace the retention policyTag

  

 #Parmamters 

 param($args1, $args2)

 $SearchIdTag = $args1

 $ReplaceTag = $args2

  

 #Lookup Function to get the GUID of the retention policy tag description

 Function ConvertIdentityToGuid($IdTag)

 {

 #Remote Session to get the exact GUID and description of the PolicyTag

    #$userCredential = get-Credential

    $username = "DOMAIN\SuperUser"

    $password = convertTo-SecureString "password" -AsPlainText -Force

    $cred = New-Object System.Management.Automation.PSCredential($username, $password)

    $session=New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://CasServer/PowerShell/ -Credential $cred

    Import-PSSession $session -AllowClobber

    $GuidTag = Get-RetentionPolicyTag -Identity $IdTag | Select GUID

    $Tag=$GuidTag.Guid

    $GuidPolicyTag = $Tag

    get-Pssession |remove-Pssession

    Write-output $GuidPolicyTag  

  

 }

  

 #Replace the tag

 Function ReplacePolicyTagOnItem($item)

 { 

   $GuidReplacePolicyTagArray = ConvertIdentityToGuid($ReplaceTag);

   $ReplaceGuidTag = $GuidReplacePolicyTagArray[1]

   #PR_POLICY_TAG 0x3019 

   $PolicyTag = New-Object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(0x3019,[Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Binary)

   $PolicyTagGUID = New-Object GUID($ReplaceGuidTag)

   $enumAlwaysOverWrite = [Microsoft.Exchange.WebServices.Data.ConflictResolutionMode]::AlwaysOverWrite

   $item.SetExtendedProperty($PolicyTag,$PolicyTagGuid.ToByteArray())

   $item.Update($enumAlwaysOverWrite)

  

  

  

  

   #Write the report to SharePoint List

   $WriteToSP = $global:AuditLog.WriteAuditLog("RetentionTagReplaced",$MailboxName,$item.Subject ,"User" ,$global:FolderName , $args1, $args2, "Item", "")

   write-host "SharePoint: " $WriteToSP

 }

  

 #Search Function 

 Function SearchPolicyTagOnItem($SearchTag)

 {

     $fvItemView = New-Object Microsoft.Exchange.WebServices.Data.ItemView(10000)

     $ItmpsPropertySet = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)

     #PR_POLICY_TAG 0x3019 

     $ItmPolicyTag = New-Object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(0x3019,[Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Binary)

     $ItmpsPropertySet.Add($ItmPolicyTag)

     $fvItemView.PropertySet = $ItmpsPropertySet;

     #Loop through  the folders

     while (($results = $ffFolder.FindItems($fvItemView)).Items.Count -gt 0) 

         {

  

         #Loop through each item in the folder

  

              foreach ($item in $results) 

              { 

                 $ItemPolicyGuid=$null;                

                 if ($item.TryGetProperty($ItmPolicyTag,[ref] $ItemPolicyGuid))

                 {                                 

                   $Itmptag = [GUID]$ItemPolicyGuid

                   if ($SearchTag -eq ($Itmptag))

                   {

                       "Item Subject: " + $item.Subject

                       "Item  Retention Policy Tag:" +$Itmptag

                       ReplacePolicyTagOnItem($item)

                   }

                 }

             } 

             $offset += $results.Items.Count 

             $fvItemView = new-object Microsoft.Exchange.WebServices.Data.ItemView(100, $offset)  

         }

 }

  

 # Script starts here

 #Global declaration

 $global:AuditLog = New-Object RetentionTagAuditLog.AuditLog

 $global:FolderName = $null

  

  

  

  

  

 #Get the GUID of the provided retention policy tag description

 $GuidPolicyTagArray = ConvertIdentityToGuid($SearchIdTag);

  

 #SharePoint assembly referenced for Reports

 [System.Reflection.Assembly]::LoadFrom("G:\PowerShell\RetentionTagAuditLog.dll")

  

 #Read data from the UserAccounts.txt.

 #Make sure this file exists in the same location as the script or provide the path

  

 import-csv AliasAccounts.txt | foreach-object 

 {

     $MailboxName = $_.EmailAddress.ToString()

     #Load the Exchange Web Service

     $dllpath = "C:\Program Files\Microsoft\Exchange\Web Services\1.1\Microsoft.Exchange.WebServices.dll"

     [void][Reflection.Assembly]::LoadFile($dllpath)

     $service = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService([Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2010_SP1)

  

     #imepersonation - need to have admin rights

     $service.ImpersonatedUserId = new-object Microsoft.Exchange.WebServices.Data.ImpersonatedUserId([Microsoft.Exchange.WebServices.Data.ConnectingIdType]::SmtpAddress,$MailboxName); 

  

     #Get current user

     $windowsIdentity = [System.Security.Principal.WindowsIdentity]::GetCurrent()

     $sidbind = "LDAP://<SID=" + $windowsIdentity.user.Value.ToString() + ">"

     $aceuser = [ADSI]$sidbind

     $service.AutodiscoverUrl($aceuser.mail.ToString(),{$true})

  

     $GuidPolicyTag = $GuidPolicyTagArray[1]

     "Checking : " + $MailboxName

     $folderidcnt = new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Root,$MailboxName)

     $fvFolderView = New-Object Microsoft.Exchange.WebServices.Data.FolderView(10000)

     $fvFolderView.Traversal = [Microsoft.Exchange.WebServices.Data.FolderTraversal]::Deep;

     $psPropertySet = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)

  

     

 

 #PR_POLICY_TAG 0x3019 

     $PolicyTag = New-Object Microsoft.Exchange.WebServices.Data.ExtendedPropertyDefinition(0x3019,[Microsoft.Exchange.WebServices.Data.MapiPropertyType]::Binary)

     $psPropertySet.Add($PolicyTag)

     $fvFolderView.PropertySet = $psPropertySet;

     $fiResult = $Service.FindFolders($folderidcnt,$fvFolderView)

  

     #Loop through each folder in the mailbox    

     foreach($ffFolder in $fiResult.Folders)

     {

         $PolicyGuid=$null;

         if ($ffFolder.TryGetProperty($PolicyTag,[ref] $PolicyGuid))

         {

  

           $ptag = [GUID]$PolicyGuid

           if ($ptag -eq $GuidPolicyTag)

           {

  

               $global:FolderName = $ffFolder.DisplayName 

               $WriteToSP = $global:AuditLog.WriteAuditLog("RetentionTagReplace",$MailboxName,"" ,"User" , $global:FolderName, $args1, $args2, "Folder", "")

  

           }

         }

         "Traversing folder: " + $ffFolder.DisplayName  

         $global:FolderName = $ffFolder.DisplayName

         SearchPolicyTagOnItem($GuidPolicyTag) 

  

     }

     # catch excption

     trap [System.Exception]

     {

      write-host ("Error: " +$_.Exception.Message)

      #write in log file

      continue

     }

 }

 

 

 

 

 


Screen Shots

 

1. Email with a retention policy tag

 

 

2.   Mapi Properties of the message item. Please see the PR_Policy_Tag

3.   Retention Policy Tag MAPI properties




4.   Run the script

 

5.   After running the script

 

6.   Verify the MAPI Properties

  

 

7.   The Retention Policy tag value has been changed

 

 

 

 

 

            
Reference

 

https://blogs.msdn.com/b/akashb/archive/2011/08/11/stamping-retention-policy-tag-using-ews-managed-api-1-1-from-powershell-exchange-2010.aspx

Comments

  • Anonymous
    January 01, 2003
    Looks like a false property name for in propTagID 0x30190102. Should be PR_POLICY_TAG.

  • Anonymous
    January 01, 2003
    This is easily one of my top 2 blogs on EWS API, and Exchange Retention.  If you have managers who are asking for you to start shifting retention policies around, you have come to the right place.  Sundar will have you talking with managers and admins authoritatively on Exchange Retention in minutes with diagrams and the key values listed out for you.  I was desperate for something more than 4 lines of code snippets that were in all of my books and Sundar delivered an enterprise solution.  This is far and above pulling subject lines from inbox mails like most of the other blogs.  Great job!

  • Anonymous
    November 15, 2011
    The comment has been removed

  • Anonymous
    January 15, 2014
    As the Exchange Retention and Archiving Policies is a topic many people run from, I think it is always

  • Anonymous
    August 08, 2014
    Great Script! Where do I pass the 'search' and 'replace' policy tags? Help?

  • Anonymous
    September 01, 2015
    Superb Article...great detailed info