The Nickname Cache
[This is now documented here: https://msdn.microsoft.com/en-us/library/ff625288.aspx ]
One of the long standing requests for Outlook development is for documentation around the format for the nickname cache. A couple months ago, development asked if I’d be interested in hosting a preview of documentation for the NK2 file. Of course, I said yes. I didn’t write this doc, but I did assist in the tech review. Down the road, you can expect to see a version of this documentation show up in the MSDN. I’ll link back to it here when it does.
This documentation applies only to Outlook 2003 and 2007 (despite mostly saying only 2007 throughout the article). There are differences in the format used by Outlook 2010 and we do expect to document them, most likely when this information is incorporated into the MSDN. For now, if you use this to mess with Outlook 2010’s nickname cache, you’re on your own.
Finally, I’ve uploaded a PDF of this document here, which includes the parsing for a sample NK2 file.
As always, let me know if you find any problems with this documentation.
[Edited 5/25/2010 to incorporate minor updates based on user feedback]
Enjoy!
Steve
Outlook 2003/2007 NK2 File Format and Developer Guidelines
This post explains how Microsoft® Office Outlook® 2007 interacts with the nickname cache file, also known as the “.nk2 file.” The .nk2 file is where Outlook 2007 persists the autocomplete list, which is the list of names that displays in the To, Cc, and Bcc edit boxes while composing an e-mail. This post also discusses the binary format of the file and the recommended ways for interacting with the .nk2 file.
This blog post should contain sufficient information to support reading and modifying the .nk2 file.
You can use any programming language to write your application because there are no dependencies on the Outlook object model or MAPI APIs.
Outlook 2007 Interaction with the .nk2 file
When a user logs on to Outlook 2007, Outlook reads the user’s MAPI Profile. Later, when the autocomplete list is shown, the .nk2 file is loaded. The .nk2 file has the same name as the profile that was used to log on, typically outlook. nk2, in the default scenario.
This file can be found here:
%APPDATA%MicrosoftOutlook
Outlook 2007 interacts with the .nk2 file in two ways:
1. Loading the .nk2 file.
2. And, later, saving the .nk2 file.
Loading the .nk2 file
Outlook 2007 loads the .nk2 file when any item with addressing functionality gets initialized. For example, e-mail addresses are used in a new mail, a mail reply, a contact item, a meeting request, etc. To load, Outlook 2007 reads all of the contents of the file as a binary stream into a structure in memory.
For autocomplete operations, Outlook interacts exclusively with this in-memory structure for the duration of the outlook.exe process lifetime. Outlook 2007 does not interact with the .nk2 file in any additional ways until it saves the in-memory structure back to disk on shutdown.
Saving the.nk2 file
Outlook 2007 saves the .nk2 file on shutdown if the autocomplete list has changed in any way.
Here are the ways that the autocomplete list gets changed:
· A new nickname entry is added through resolving a name, picking a recipient from the address book dialog, or sending mail to a recipient that was not already in the list.
· An entry is modified by sending mail to an existing recipient in the list.
· An entry is removed by the user through the UI.
· Other minor scenarios not relevant to this blog post
Outlook 2007 does not save the .nk2 file on shutdown if the autocomplete list has not changed. The save involves writing the in-memory structure to the .nk2 file as a binary stream.
If the size of the .nk2 file contents shrink during an Outlook session (for example, the user deletes some entries), Outlook saves the new contents to the file, but the file itself will not shrink in size.
Recommendations
· Never partially modify the .nk2 file. The supported interaction is to 1) read the entire file into memory, 2) modify the memory structure, and 3) write out the entire file when the modifications are finished.
· We recommend locking the file from modification by other processes while you’re reading it and writing it using standard Windows file locking APIs (e.g. LockFile in C/C++ and FileStream.Lock in C#).
· Don’t interact with the .nk2 file while Outlook is running. If Outlook is running while you modify the file, Outlook will likely overwrite your changes when it shuts down.
· Do not write properties of type PT_MV_UNICODE and PR_MV_STRING8 into an .nk2 file to be consumed by Outlook 2003. These properties are only understood by Outlook 2007.
· Do not write properties of types that are not mentioned in this document.
NK2 File Format
In addition to knowing how Outlook interacts with the .nk2 file, you must also understand the binary file format of the .nk2 file.
The .nk2 file is a set of recipient property rows that are saved as a binary stream along with some bookkeeping metadata that is used only by Outlook 2007 and Outlook 2003.
The metadata is relevant to Outlook’s interactions with the .nk2 file sothird parties must preserve what is in each metadata block when saving a modified .nk2 file. In other words, third parties should modify only the row-set portion of the binary format and preserve what was already in the metadata blocks of the file.
When creating a new .nk2 file from scratch, use the metadata values from the binary example to populate the metadata in the new file.
File Visualization
The high-level layout of the file looks like this:
Metadata (12 bytes)
Number of rows n (4 bytes)
Number of properties p in row one (4 bytes)
Property 1’s property tag (4 bytes)
Property 1’s reserved data (4 bytes)
Property 1’s value union (8 bytes)
Property 1’s value data (0 or variable bytes)
… (property 2 through property P-1)
Property p’s property tag (4 bytes)
Property p’s reserved data (4 bytes)
Property p’s value union (8 bytes)
Property p’s value data (0 or variable bytes)
Number of properties q in row two (4 bytes)
… (row two’s properties)
… (row 3 through row n-1)
Number of properties r in row n (4 bytes)
… (row n’s properties)
Metadata (12 bytes)
High-level Layout
Broadly speaking, this is the layout of the .nk2 file:
Value Data |
Number of Bytes |
Metadata |
12 |
Row-set |
Variable |
Metadata |
12 |
Row-set Layout
The row-set layout is as follows:
Value Data |
Number of Bytes |
Number of rows |
4 |
Rows |
Variable |
The number of rows identifies how many rows will come in the next part of the binary stream sequence.
Row Layout
Each row is of the following format:
Value Data |
Number of Bytes |
Number of properties |
4 |
Properties |
Variable |
The number of properties identifies how many properties will come in the next part of the binary stream sequence.
Property Layout
Each property is of the following format:
Value Data |
Number of Bytes |
Property Tag |
4 |
Reserved Data |
4 |
Property Value Union |
8 |
Value Data |
0 or variable (depending on the prop tag) |
Interpreting the Property Value
The Property Value Union and the Value Data are to be interpreted based on the property tag in the first 4 bytes of the property block. This property tag is in the same format as a MAPI property tag. Bits 0 through 15 of the property tag are the property’s type. Bits 16 through 31 are the property’s identifier. The property type determines how the rest of the property should be read.
Static Values
Some properties have no Value Data and only have data in the union. The following property types (which come from the Property Tag) should interpret the 8-byte Property Union data as follows:
Prop Type |
Property Union Interpretation |
PT_I2 |
short int |
PT_LONG |
long |
PT_R4 |
float |
PT_DOUBLE |
double |
PT_BOOLEAN |
short int |
PT_SYSTIME |
FILETIME |
PT_I8 |
LARGE_INTEGER |
Dynamic Values
Other properties have data in a Value Data block after the first 16 bytes that contain the Property Tag, the Reserved Data, and the Property Value Union. Unlike static values, the data stored in the 8 byte Property Value union is irrelevant on reading. When writing, make sure to fill these 8 bytes with something, but the content of the 8 bytes doesn’t matter. In dynamic values, the property tag’s type determines how to interpret the Value Data.
PT_STRING8
Value Data |
Number of Bytes |
Number of bytes n |
4 |
Bytes to be interpreted as an ANSI string (includes NULL terminator) |
n |
PT_UNICODE
Value Data |
Number of Bytes |
Number of bytes n |
4 |
Bytes to be interpreted as a UNICODE string (includes NULL terminator) |
n |
PT_CLSID
Value Data |
Number of Bytes |
Bytes to be interpreted as a GUID |
16 |
PT_BINARY
Value Data |
Number of Bytes |
Number of bytes n |
4 |
Bytes to be interpreted as a byte array |
n |
PT_MV_BINARY
Value Data |
Number of Bytes |
Number of binary arrays X |
4 |
A run of bytes containing X binary arrays. Each array should be interpreted like the PT_BINARY byte run described above. |
Variable |
PT_MV_STRING8 (Outlook 2007)
Value Data |
Number of Bytes |
Number of ANSI strings X |
4 |
A run of bytes containing X ANSI strings. Each string should be interpreted like the PT_STRING8 byte run described above. |
Variable |
PT_MV_UNICODE (Outlook 2007)
Value Data |
Number of Bytes |
Number of UNICODE strings X |
4 |
A run of bytes containing X UNICODE strings. Each string should be interpreted like the PT_UNICODE byte run described above. |
Variable |
Significant properties
As mentioned above, the binary blocks that represent properties have property tags that correspond to properties on address book recipients. For properties that aren’t listed here, you can look up the property description at https://msdn.microsoft.com/en-us/library/cc433490(EXCHG.80).aspx.
The properties below are the minimum set of properties necessary for a row to be valid. Therefore, new rows added to the .nk2 file must be populated with the properties below.
Property Name |
Property Tag |
Description (see MSDN for more details) |
PR_NICK_NAME_W (not transmitted on recipients, specific to .nk2 file only) |
0x6001001F |
This property must be first in each recipient row. Functionally serves as a key identifier for the recipient row. |
0x0FFF0102 |
The address book entry identifier for the recipient. |
|
0x3001001F |
The recipient’s display name. |
|
0x3003001F |
The recipient’s e-mail address (e.g. johndoe@contoso.com or /o=Contoso/OU=Foo/cn=Recipients/cn=johndoe). |
|
0x3002001F |
The recipient’s address type (e.g. SMTP or EX). |
|
0x300B0102 |
The recipient’s MAPI search key. |
|
0x39FE001F |
The recipient’s SMTP address. |
|
| PR_OBJECT_TYPE | 0x0FFE0003 | Represents the type of this recipient. Values can either be MAPI_MAILUSER or MAPI_DISTLIST. |
| PR_DISPLAY_TYPE | 0x39000003 | Similar to PR_OBJECT_TYPE, but it’s used by Outlook’s UI to determine how to display the recipient (for example, bolding a distribution list). Common values for this are DT_DISTLIST and DT_MAILUSER. |
| PR_NEW_NICK_NAME (not transmitted on recipients, specific to .nk2 file only) | 0x6002000B | Specifies whether this row was just added to the nickname cache or not. If you are creating a new row, this should be set to true. |
PR_DROPDOWN_DISPLAY_NAME_W (not transmitted on recipients, specific to .nk2 file only) |
0x6003001F |
The display string that shows up in the autocomplete list. |
PR_NICK_NAME_WEIGHT (not transmitted on recipients, specific to .nk2 file only) |
0x60040003 |
The weight of this autocomplete entry. The weight is used to determine in what order autocomplete entries show up when matching the autocomplete list. Entries with higher weight will show before entries with lower weight. The entire autocomplete list is sorted by this property. The weight periodically decreases over time and increases when the user sends an e-mail to this recipient. See the description below for more information about this property. |
PR_NICK_NAME_WEIGHT
The set of rows in the .nk2 file is sorted in descending order by the PR_NICK_NAME_WEIGHT property and the .nk2 file should always preserve this sorted characteristic. Therefore, any changes to a row’s weight should also ensure that the row’s position retains the sorted order of the entire set of rows. Any additions to the row-set should be inserted to the proper position to maintain the sorted order.
The minimum value of this weight is 0x1 and the maximum value is LONG_MAX. Any other values for the weight are considered invalid.
When Outlook 2007 sends a mail to or resolves a recipient, it will increase that recipient’s weight by 0x2000.
Comments
Anonymous
March 15, 2010
Thanks Steve. Just a couple of typos: Under Interpreting the Property Value, the doc should say "Bits 0 through 15" instead of bytes. Same for the bits 16 through 31.Anonymous
March 15, 2010
Thanks Lev - fixed it!Anonymous
April 20, 2010
Is there supposed to be a complete list of the property tag values and their associated names, etc. in the document you have linked to underneath Significant Properties? I can't seem to find them in there such as the PR_NICK_NAME_W property with tag value = 0x6001001f...Anonymous
April 20, 2010
I don't follow your question. PR_NICK_NAME_W is defined above. What are you looking for that you cannot find?Anonymous
April 20, 2010
You mention that you can find property descriptions for the properties not in your list in the documentation you have linked in the text below your header 'Significant Properties.' I can't seem to find a list that includes the ones you have listed... I'm sure I'm just missing something simple...Anonymous
April 20, 2010
That link was just intended as a reference to any properties you may find in an .n2k file that aren't listed here.Anonymous
October 12, 2010
Well - since Outlook uses this, and overwrites it upon change, and 99% of the time for anyone needing to play with this - outlook will be running... how do we instruct outlook itself to modify it's list?Anonymous
October 12, 2010
Chris - maybe you'd rather we didn't document it? Outlook doesn't have an API for modifying the file so this is the only way to do it programmatically. You can use the UI to add/remove entries from the cache though.Anonymous
November 23, 2011
Hello, I'm trying to find a way of saving my autocomplete list to the contacts folder, so that I may then export them as a vcf. I must no use a work computer that does not have outlook (nor an I allowed to install it.) I am planning on using thunderbirds and would like to find an acceptable way to bring in the NK2 file once saved as vcf. Is this possible. Please note, I am not a programmer, just a user! thanks