Conversation history from O365 Lync into Outlook suddenly stops

Whilst looking into Lync O365 issues, sometimes we cross the "border" into general exchange issues.  The whole service is known as unified communications so I can't be too surprised when I start to look into exchange.

The symptoms of the issue I will blog / talk today are:

• Conversation history does not appear in the Conversation History folder within Outlook 2010 or OWA.
• Issuing an "outlook.exe / resetfolders", does open Outlook and around 10 messages from the conversation history from Lync is sync'd (and this is replicated up to OWA), but no further are moved.
• Using Lync on OWA results in the same symptoms.

Lync uses MAPI or EWS to communicate between itself and Outlook / Exchange. EWS is the preferred method of communication, and MAPI can be used but with reduced functionality.  For more details I encourage you to read this document,  Microsoft Lync Server 2010 - Understanding and Troubleshooting Microsoft Exchange Server Integration.  But for now, have a look below for an extract:

Many of the features offered within Lync 2010 have dependencies on connectivity to a Microsoft Exchange mailbox. For example, the conversation environment feature leverages both Exchange Web Services (EWS) and MAPI to manage Conversation History items. Unlike previous versions of Lync, EWS is now the primary method used to provide Microsoft Exchange integration features for the Lync client. MAPI will be used if EWS is unavailable, but only in a limited capacity.

The impact of this design can be significant for a given user, depending on the location of their mailbox. If the user has an Exchange Server 2003 mailbox, the Lync client will not be able to use EWS to interact with Microsoft Exchange. Rather, MAPI will be used in a fallback capacity to access the Microsoft Exchange mailbox. While the Lync client will be able to write new archived Conversation History items into the user’s mailbox, it will fail to retrieve any existing Conversation History items from the user’s mailbox.  

However, for a user whose mailbox resides on a server running Exchange Server 2007 SP1 or Exchange Server 2010, the Lync client will leverage EWS to provide the user with a complete experience for the conversation environment. Not only will EWS be used to write new archived Conversation History items into the user’s mailbox, but EWS will also be used to retrieve and display past Conversation History items to the user in the Lync client.

Thats pretty clear.  For those who want to check on EWS & MAPI on their own client, can hold CTRL + Right Click on the Lync icon on the tool tray and select "Configuration Information".  That will give you a screen shot of the status and settings for your Lync client:

So, back to the issue at hand.  Looking at it, first I would have send we have a EWS connectivity issue.  Looking at the HTTP logs from Fiddler I would say thats not the problem.  The 200's were returning fine.

So its time to delve into the logs for the Lync client.  The first port of call for this kind of issue is the ETL log for communicator.  Have a look here if you want to know how to gather and view those logs.

Diving in the ETL, we had to locate the point at which the client tries to retrieve or write to the folder as well as seeing if the history is actually being recorded from the Lync client.

The Lync client stores the conversation history in a spooler located in C:\Users\leoncon\AppData\Local\Microsoft\Communicator\sip_myuser@contoso.com\History Spooler.   The files themselves have a .hist file extension.  Checking there I could see that there were many waiting to be sent to the exchange server for archiving.  So was the connection being established by EWS ok?  The clients configuration information reported that to be the case.

Looking into the ETL we saw errors being returned by the FindFolder response in Lync.  Thanks to our friends across the pond we had found the cause of the conversation folder not being found, and the theory went that since it could not be found it could not be written too.  So it looks like the Lync client issues a SOAP request for the Conversation History folder & the Missing Conversations folder.  In with that is a request for a list of folders in the exchange store.

The key ETL entry was this:

Component: (Shared)
Level: TL_ERROR
Flag: OC_MAPI_EWSFOLDERMANAGER
Function: NModel::CEwsMailboxFolderManagerTask::CheckEwsResponseCode
Source: EwsMailboxTasks.cpp(4281)
Local Time: 06/07/2012-17:09:02.859
Sequence# : 0002411E
CorrelationId :
ThreadId : 2730
ProcessId : 2868
CpuId : 3
Original Log Entry :
TL_ERROR(OC_MAPI_EWSFOLDERMANAGER) [3]2868.2730::06/07/2012-16:09:02.859.0002411e ((Shared),NModel::CEwsMailboxFolderManagerTask::CheckEwsResponseCode:EwsMailboxTasks.cpp(4281))<O_TRC><ADR>0x0C6776D0</ADR>Found error in CEwsMailboxFolderManagerTask response. responseCode=ErrorExceededFindCountLimit. reportedStatus=ErrorExceededFindCountLimit.</O_TRC>

The ETL also logged the XML response from the EWS service to this request.  Here is an extract:

You can see from the image that the conversation history folder is retrieved ok. Is just the next response that errors.  The important part in this message is the PolicyLimit.  This defines the number of folders that can be returned by the EWS service before an error is thrown.  At the moment it is set to 1000.  So the error in this SOAP response prevents the Conversation History folder from being retrieved. 

Next the ETL showed that the client still attempted to create a new version of the Conversation History folder via EWS.  That was also returned with an error that the folder already exists and no further action will be taken.

So there is a problem with folder count.  Or is there?  We need to check if the folder count is greater than 1000.  Obviously counting by eye is an option, but not practical.  The properties of the folder (from Outlook), shows size, but not number of folders.  So I found this VBA script to count the folders through an Outlook macro:

Sub CountFoldersInMBX()

Dim outapp As Outlook.Application
Set outapp = CreateObject("Outlook.Application")
Dim olns As Outlook.NameSpace
Set olns = outapp.GetNamespace("MAPI")
MsgBox "Total: " & GetSubFolderCount(olns.GetDefaultFolder(olFolderInbox).Parent)

End Sub

Function GetSubFolderCount(objParentFolder As MAPIFolder) As Long
Dim currentFolders As Folders
Dim fldCurrent As MAPIFolder

Set currentFolders = objParentFolder.Folders
If currentFolders.Count > 0 Then
Set fldCurrent = currentFolders.GetFirst
While Not fldCurrent Is Nothing
TempFolderCount = TempFolderCount + GetSubFolderCount(fldCurrent)
Set fldCurrent = currentFolders.GetNext
Wend
GetSubFolderCount = TempFolderCount + currentFolders.Count
Else
GetSubFolderCount = 0
End If
End Function

You will need to run this code using this process:

1. In outlook go to File > Options > Customise Ribbon
2. On the right hand tick list box, Developer should be un-ticked, tick Developer and click ok.
3. A new tab should appear on the outlook ribbon at the top called “Developer”
4. Click on this, then click on “Visual Basic”
5. A new window should appear.  Open up Project1 on the tree view on the left hand side.
6. Double click on ThisOutlookSession, a blank page should open.
7. Copy and paste the code above into this page.
8. Click on the play button on the top of the main screen.
9. Click on Run on the next screen.

This code I got from a blog from one of our other employees here.  As soon as I find the link again I will have to amend this post to include.

The customer though had 1000+ folders. Once the customer had amalgamated their folders and brought the count down to under 1000 folders the issue went away.  The customer issued an "outlook.exe /resetfolders" and over time the conversation folder filled up with a past history.  Problem solved. 

As for the future, I have been told that the next release of Lync will resolve this issue.  Quite when that will happen I don't know.  This issue has been resolved and its one that is weird enough for me to blog about. Please feel free to comment!

Comments

• Anonymous
  January 18, 2013
  Microsoft Lync Console you can find 3 icons 1st Contacts, 2nd Activity Feesds , 3rd Conversations. Click on 3rd icon  Conversations. At bottom of the console you can find "View More in Outlook" Click on it and find all the conversation history folder in outlook.

• Anonymous
  April 01, 2013
  How to Save Office Communicator IM Conversation to My excel file using excel vba macros.