CRM callouts are just plain hard to write

Building callout handlers for MS-CRM is hard. It’s just plain hard. Not because writing a COM object that implements an interface is hard – we’ve got a boatload of tools to help with that. They’re hard to write because the model is inconsistent and incomplete. The v1.x callout interface is very simple and has very simple semantics. It’s this simplicity that makes it useful, powerful, and hard to use.

The callout interface looks like so. You’ve seen it in the documentation, and probably tried to implement it somewhere along the way. You might have even succeeded.

[

    uuid("F4233E5B-17DC-4661-9ABC-6707A9F99215"),

    dual

]

interface ICRMCallout : public IDispatch

{

    HRESULT PostCreate([in] int ObjectType, [in] BSTR ObjectId, [in] BSTR OrigObjectXml);

   HRESULT PostUpdate([in] int ObjectType, [in] BSTR ObjectId, [in] BSTR OrigObjectXml);

    HRESULT PostDelete([in] int ObjectType, [in] BSTR ObjectId);

};

In case you’re a C# person, here’s the definition from the SDK. There’s not much difference.

[GuidAttribute("F4233E5B-17DC-4661-9ABC-6707A9F99215")]

public interface ICRMCallout

{

    Int32 PostCreate(Int32 ObjectType, String ObjectId, String OrigObjectXml);

    Int32 PostUpdate(Int32 ObjectType, String ObjectId, String OrigObjectXml);

    Int32 PostDelete(Int32 ObjectType, String ObjectId);

}

The basic semantics are as follows:

· PostCreate provides the handler with the instance XML as supplied by the platform consumer (this could be the application, integration, or another callout). The first parameter is self-explanatory, as is the second. OrigObjectXml on the other hand deserves some explanation and some discussion. I’ll get to that in a bit because it applies to PostUpdate as well.

· PostUpdate follows the same pattern but happens after the changes have been submitted to the database. Note that I said “submit” and not “commit”. There’s an important distinction and this is one of those incomplete things about the callout interface. This problem applies to PostCreate as well so I’ll talk about it in a bit.

· PostDelete is the simplest to understand and one of the hardest to use. It’s fired after a soft delete request is made to an instance. The only information supplied is an instance identifier (the type code and id).

 

Side note – if you’re implementing a callout handler in C# you need to declare your class as follows. The thing that seems to be one of the biggest PSS issues is the GUID. It is a requirement that you add your own GUID. Don’t use the one from the documentation because someone else might have made the same mistake and now there are two registered COM classes implementing the same interface. Well, that’s not entirely true, there’s one – the last one to get registered.

 

[ClassInterface(ClassInterfaceType.AutoDispatch)]

[GuidAttribute("put a CLSID here")]

public class MyCallout : ServicedComponent, ICRMCallout

 

Submit vs. Commit

Transaction semantics are not well-defined for callouts. Sometimes the callout is made inside of a transaction and sometimes it’s made outside. The sometimes isn’t well understood on the CRM team right now. That’s one of the problems with building large software systems from the ground up with a growing team. But that’s a discussion for another time. The important bit is that the transaction rules around callouts are ill-defined and the only assumption a callout author can make is that the instance in question is likely inaccessible for the duration of the callout function execution.

What does this mean? Well, first it’s important to know how the platform calls the handler class. Let’s start by looking at the basic flow.

 

    COSERVERINFO si;

    si.dwReserved1 = 0;

    si.dwReserved2 = 0;

    si.pAuthInfo = NULL;

    si.pwszName = wszComputerName;

    MULTI_QI qi = { pIID, NULL, 0 };

    hr = CoCreateInstanceEx( callout, NULL, CLSCTX_REMOTE_SERVER, & si, 1, &qi );

The thing to notice is that the platform is using an explicit out-of-proc call. Why? Because we really don’t want the platform to crash if the callout crashes. This means that the callout handler must be registered as an out-of-proc server, which is easily done using a COM+ application. But that also means that callouts can’t just be dropped on the server machine and registered as COM objects.

Let’s look at how the platform actually calls a handler.

for each pCallout

{

   // call the event handler

    pCallout->PostCreate(otc, bstrId, bstrXml);

    // then release the interface pointer

    pCallout->Release();

}

The platform will walk the list of registered callouts and for each one it finds, and can create, it’ll call. But isn’t the definition of PostCreate supposed to return an HRESULT? Yup, and it’s being ignored by the platform because there’s no clear answer about what should occur if a given callout fails. This is good and bad. It’s good because the platform doesn’t get mired down in the transactional details necessary to “fix” things that might break in an unknown chunk of code. It’s also terribly bad for the same reason.

Oh yeah, and all this happens deep, deep, deep in the platform infrastructure. So deep that we might as well consider them the equivalent of a database trigger. Not that triggers are bad or anything, many GP ISVs have built all kinds of solutions by adding triggers to the GP database. They’re bad because they happen to look like a trigger, and happen to behave like a trigger, but just aren’t triggers. There’s just no trigger context available. One thing that makes them very much like triggers is that they happen for every WRITE (for example) and WRITEs happen all the time in the platform for reasons that callout authors usually don’t care about (like on a security descriptor update because someone changed a role – does the callout care, probably not).

The next thing to notice about the callout is that the platform calls it inline with the rest of the business logic happening on the current thread. This means that your customer (the application user typically) is sitting there patiently waiting for your callout code to complete. If you’re making a long-lived call to another application the user is effectively blocked from getting any other work done. It also means that the platform is blocked. At least one critical resource is waiting: the thread servicing the user request. But there may be other resources blocked depending on which platform call was made: database resources.

Why does this matter?

Well, if the database is blocked because the platform is in the middle of a transaction it means that other callers can’t get at the blocked resource. That caller might very well be your callout handler if it needs to call back into the platform to retrieve data. And, if the callout author is well-behaved then the callback to the platform is happening over SOAP, and that’s clearly an out-of-proc call.

This is what I meant by incomplete earlier. The callout only gets the information that was supplied by the original caller and this data is clearly incomplete. The auditing, owner, and default data is missing from the XML for PostCreate, and it’s probably old data for a PostUpdate. The way around this is simple: call back into the platform to query the rest of the data. Oops, now we’re in trouble because we’ve probably gotten ourselves into a deadlock situation, and that’s not a good thing.

There’s also the issue of PostDelete. Normally the delete handler works fairly well. It gets called when something gets deleted and it gets enough information to do something about it. Let’s say that a Sales Order Detail was deleted though. The callout will get the line item instance ID and nothing else. How should it deal with this? There isn’t a story. You could cheat and make an ADO call into the database to try to read the Sales Order ID from the just-deleted line item, but this is bad because we don’t want you reading the database directly (because we can and will change the structures from release to release) and that pesky deletion service might beat you to it and really delete the instance. Like I said, callouts are incomplete and themselves don’t have enough context to help the callout author.

Uh, then what?

Given all that, what’s a callout coder to do? First, make the callout hander as short, fast, and simple as possible. I recommend converting the callout parameters, along with some general contextual data, into a message and dropping it in a queue somewhere. This doesn’t have to be anything fancy. It can be MSMQ, a database table (in another database please), or even in the file system. This way control is returned to the platform as soon as possible which means control is returned to the user sooner. The next thing is to use a service (or other application) to watch that queue and do the expensive work out-of-band. This service is just another platform user and can call into the API to read the rest of the necessary data. Well sort of.

Remember when I mentioned incomplete (yeah, I think I’m starting to repeat myself here). Sure, not getting the complete instance data is a problem, and if that problem were solved we could mitigate a lot of the call-back-into-the-platform problems. But what’s really missing here is the contextual information necessary to know what to do about the callout. For example, who made the original platform request? Was it an application user or was it your own callout code? There is no way to know. Part of protecting the platform from callout crashes means that getting context (current transaction, current method, and current user) to the callout is really difficult.

For now we’re stuck. We can’t change the interface without breaking all those people who’ve already started writing handlers. We can add a new interface, but it’ll still suffer from a lot of the same problems. We need to go back to the drawing board and rethink how all this stuff can and should work, where it might best fit, how the ISV community might use it, and what tools should be supported (i.e. COM? VB6? .Net?)

If you’ve read this far looking for enlightenment, I’m sorry, I don’t have the answer. I’ve told you what’s wrong with callouts, but you probably already knew that. Though I have given a little insight into how they work and I’m hoping that helps you understand how to write handlers that behave well. But, for now, building callout handlers for MS-CRM is hard.

Comments

• Anonymous
  December 09, 2004
  Is there a next version coming where this will get better?
• Anonymous
  December 09, 2004
  I can't really comment on what's coming in future releases, as much as I'd like to. Things just change too fast - one minute a feature might be in, one minute it might out. If I said it was going to get better, then someone would decide that we need to focus on other functionality and cut the feature.
  
  I can say that there is a next version (it's currently called MS-CRM 2005) and that the team knows how hard callouts are.
• Anonymous
  December 09, 2004
  So there really is a window between the "PostDelete" trigger and when it is REALLY deleted from the Database?
  
  I am using PostCallouts for a custom integration with another system, and need to "fetch" a custom "ExternalID" field so I know which record to delete in the other system.
  
  I tried going "under the sheets" and getting the data using straight ADO, but the data was already gone. If I use the SDK methodology to retrieve it immediately, should it be there??
  
  Thanks!
• Anonymous
  December 09, 2004
  The comment has been removed
• Anonymous
  January 03, 2005
  The comment has been removed
• Anonymous
  January 06, 2005
  I found this article using a Google search to try to find more information on callouts in CRM and it is very informative--thanks for taking the time to clarify some of the sparsely documented details of what could be a very powerful feature of CRM, Mike. What I was really trying to find out, though, is what information is contained in OrigObjectXml? Is it the original object data (from before any changes were made), or the object data that was submitted?
• Anonymous
  January 06, 2005
  It's exactly the data that was supplied on entry to the API. That's one of the things that makes it fairly useless in its current design. The original design called for POST actions to contain the data in its POST action state - that is, all the business logic, defaulting, and database logic would have been applied and the entity would have appeared as if you'd read it through the API. The PRE actions were to have the data as it came through the API and as it had been modified by other upstream callouts. Well, PRE never happened, and POST behaves in a half-baked way today.
  
• Anonymous
  April 14, 2006
  
  I've been looking into bridging our CRM 3.0 installation with the billing section of Quickbooks so we can do simple imports and exports of data between the two. So far, as far as I can tell, this might be more work than I initially suspected. Although
• Anonymous
  April 14, 2006
  I spent yesterday feeling your pain. I needed to create a V3-compatible callout handler that would capture all events for all types.
  
  First, let's just say the metadata is your friend (like I haven't said that before). I hacked a little SQL script to generate the full callout.config XML file for everything. Sure saves typing XML (yuck!).
  
  Next, I did the "right thing" and made sure I had MSMQ installed and configured. My little callout handler, which has to be written in CLR 1.1 simply catches the event, wraps it up all pretty-like, and slams it into a queue. On the other end is a beefy CLR 2.0 callout event processor that can do all kinds of nice stuff.
  
  Note that this is all post-callout stuff. My feeling is that pre-callouts should be exposed as declarative workflow-based "code". That gets us out of the problem with making the user wait for a handler to handle, and it saves the developer from having to suffer through the mess that is callout transactions.