Rude Operator

If you are automating Outlook in a standalone application or other app outside of Outlook and have trouble on some machines with your CreateObject call succeeding when Outlook is not already running, this article may be for you.

Have you ever called someone only to be put on hold as soon as they answer? Do you hang up immediately, or do you hold on for a few minutes to see if they pick up? If you’ve ever ordered a pizza where I order mine, I know you’ve experienced this. Well, it turns out, Outlook [or any Office application for that matter] acts just like this rude operator. When you try to automate Outlook, the Outlook.exe process has to be running in order to serve your requests. Outlook is what’s called a COM server. So your call to CreateObject (or "new”) makes a call into the COM subsystem which does the work of spinning up an instance of Outlook for you to work from using the –Embedding switch which tells Outlook not to display any UI by default. Part of Outlook’s start up logic involves loading all the COM add-ins and calling into the OnConnect and OnStartupComplete events. This is where things could go badly for you.

If your user is an comaddinophile they may have some add-ins installed that take a looooooong time to load. They may be calling into a database or a webservice or just have really poor code that takes a long time to execute. Outlook does all this work on its main thread – they aren’t spun off to a background thread – so all the time it takes for these add-ins to load and run is time the COM subsystem is spending growing very impatient. It can, after a while, just time out or it could succeed the initial load, but if Outlook is busy handling other requests your call to get the Application dispatch object could be actively refused because Outlook is blocking. In these cases, you could end up with a RPC_E_SERVERCALL_RETRYLATER error bubbling up in your .net automation code. The error you get will be something like “Call was rejected by callee.”

Oftentimes, you can follow up your failed CreateObject call with a second one and it succeeds because at this time Outlook.exe is running and may be unblocked and ready to service your request. However, it may just block again. In these cases, you will need to implement an IMessageFilter to be able to tell the COM subsystem that you’re ok waiting and to keep retrying your call until it succeeds or you get a different error.

The best sample I found for this was describing implementing this to solve this same problem when doing Visual Studio automation, but the same logic applies. Andrew Whitechapel wrote a sample also. In his case he was describing a similar problem when you try to make object model calls on a background thread from an add-in. Here’s a VB.NET sample I wrote.

Public Class MessageFilter
    Implements IOleMessageFilter

    Public Shared Sub Register()
        Dim newFilter As IOleMessageFilter = New MessageFilter()
        Dim oldFilter As IOleMessageFilter = Nothing
        CoRegisterMessageFilter(newFilter, oldFilter)
    End Sub

    Public Shared Sub Revoke()
        Dim oldFilter As IOleMessageFilter = Nothing
        CoRegisterMessageFilter(Nothing, oldFilter)
    End Sub
    Public Function HandleInComingCall(ByVal dwCallType As Integer, ByVal hTaskCaller As System.IntPtr, ByVal dwTickCount As Integer, ByVal lpInterfaceInfo As System.IntPtr) As Integer Implements IOleMessageFilter.HandleInComingCall
        Return 0
    End Function

    Public Function MessagePending(ByVal hTaskCallee As System.IntPtr, ByVal dwTickCount As Integer, ByVal dwPendingType As Integer) As Integer Implements IOleMessageFilter.MessagePending
        Return 2
    End Function

    Public Function RetryRejectedCall(ByVal hTaskCallee As System.IntPtr, ByVal dwTickCount As Integer, ByVal dwRejectType As Integer) As Integer Implements IOleMessageFilter.RetryRejectedCall
        If (dwRejectType = 2) Then
            Return 99
        Else
            Return -1
        End If
    End Function

    <DllImport("Ole32.dll")>
    Private Shared Function CoRegisterMessageFilter(ByVal newFilter As IOleMessageFilter, ByRef oldFilter As IOleMessageFilter) As Integer
    End Function

End Class

<ComImport(), Guid("00000016-0000-0000-C000-000000000046"),
    InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIUnknown)>
Public Interface IOleMessageFilter
    <PreserveSig()>
    Function HandleInComingCall(
                               ByVal dwCallType As Integer,
                               ByVal hTaskCaller As IntPtr,
                               ByVal dwTickCount As Integer,
                               ByVal lpInterfaceInfo As IntPtr) As Integer

    <PreserveSig()>
    Function RetryRejectedCall(
             ByVal hTaskCallee As IntPtr,
            ByVal dwTickCount As Integer,
            ByVal dwRejectType As Integer) As Integer

    <PreserveSig()>
    Function MessagePending(
            ByVal hTaskCallee As IntPtr,
            ByVal dwTickCount As Integer,
            ByVal dwPendingType As Integer) As Integer

End Interface

To use it, just drop that in your code and then wrap your automation code like this:

Private Sub Automate()

    MessageFilter.Register()

    Try

        Dim myOutApp As Outlook.Application = New Outlook.Application()
        Dim myOutNS As Outlook.NameSpace = myOutApp.Session
        Dim myOutlookFolder As Outlook.MAPIFolder

        myOutlookFolder = myOutNS.GetDefaultFolder(Outlook.OlDefaultFolders.olFolderInbox)

        MessageBox.Show(String.Format("{0} unread items in Inbox.", myOutlookFolder.UnReadItemCount))

        System.Runtime.InteropServices.Marshal.ReleaseComObject(myOutlookFolder)
        System.Runtime.InteropServices.Marshal.ReleaseComObject(myOutNS)
        System.Runtime.InteropServices.Marshal.ReleaseComObject(myOutApp)

    Catch ex As Exception

        'Handle Exception

    Finally

        MessageFilter.Revoke()

    End Try
End Sub

The important part to notice is the calls to MessageFilter.Register and MessageFilter.Revoke.