Share via

Convoys

  • Article

CONVOYS

 

From the BizTalk Documentation:

When a group of correlated messages could potentially be received at the same time, a race condition could occur in which a correlation set in a particular orchestration instance must be initialized by one of the messages before the other messages can be correlated to that orchestration instance. To ensure that all of the correlated messages will be received by the same orchestration instance, BizTalk detects the potential for such a race condition and treats these messages as a convoy.

Since I helped proofread the docs (and write this paragraph), I figure I can cut and paste some of them. I spent a while at teched talking to my friend Marty about convoys and what they really are so lets see if we can get this message broadcast out there. Convoy processing is not a feature of BizTalk which you should be looking at and saying, “hmmmmm. I wonder how I can use this convoy thing.” A convoy is “something we support”. Convoy is a term which we use to describe a class of application protocols, specifically it is a set of application protocols which have a race condition as described above. Let’s take an example. Say you are a hospital and want to have a service which handles all information about each patient. For a given patient you have three types of messages, an admittance message, status messages, and a discharge message. If you look at your protocol, you will have built a service which just receives. Now let’s think about what could happen. Let’s say you send the patient admittance message and it goes through the system using maybe a synchronous protocol like HTTP. That means when you get the 202 back, you know the message has been delivered. But what if the BizTalkServer host which is actually supposed to process the message hasn’t started yet (ie the nt service is stopped). Maybe you had a power outage, maybe some intern decided to “hit this button”, who knows. So the orchestration instance which is supposed to handle all messages for patient X has not physically started. The message is in the database (MessageBox) and if you look in HAT you will see the orchestration is marked as ready to run, but it can’t start cause there is no where for it to start. Now lets say you send a patient update. Let’s think about what BizTalk could be doing (okay actually is doing) to support general correlation. So when a message initializes a correlation set, the service instance which the message is delivered to has the responsibility of communicating to the underlying pub/sub system (the MessageBox) that it is now expecting subsequent messages which will follow the same correlation set (based upon what other receive actions you have with follow the same correlation set). In a simple conversation style protocol (ie, you say “Hi”, I respond by saying “Howz it goin’”, and then you start talking), I can create all required subscriptions for subsequent messages when I do the next send which follows that correlation set. The protocol indicates that the original sender will not send any more messages until he gets the response back from me acknowledging the start of our conversation. There is no race. I transactionally create the subscriptions at the same time as sending the next message which acknowledges our conversation so you can’t possibly send a message which I am not ready for. Again, there is no race here. Now lets look back at my original convoy scenario. As you can see, the service instance which will handle all data for that patient does not communicate back to the sender. Heck, it might not even really know who the sender is. So there is this race, since you cannot depend on the service instance to be created and have a chance to turn around and create the subscription before subsequent messages are sent. *We solve this*. When you think of convoys, I want you to not think of BizTalk features, but rather think of your application / business protocols and say, “Hey that’s a convoy. That’s alright though, BizTalk will detect it and handle it and it will just work.” Convoy is not something you try to use, it is something you just end up using because that is how your business works and we support it. To give you a bit of insight, what happens is that our compiler will detect this race condition and communicate it down to the messagebox at enlistment time. Normally enlistment just creates the activation subscriptions (subscriptions which start the new instances), but in this case it will actually create some sort-of templated subscriptions and link them together around this convoy set. Hence, when the first message comes in, the data which coincides with the convoy set is pulled and we essentially complete the templated subscriptions so that any subsequent messages which match those subscriptions and have the same convoy set properties will go to the newly created instance, even if it hasn’t started yet.

So what is the real difference between a parallel and sequential convoy? Okay, well there are a number of restrictions imposed by the orchestration runtime which basically require your protocol to make sense, but if you want to know the real guts of it, all it really is is telling the pub/sub infrastructure which of these templated subscriptions can initialize the convoy and which require it to already be filled. So if you think about, when the first message comes to the activation subscription, it has to fill out the convoy set for its specific data. Then subsequent messages which match other subscriptions will find there data already filled and so go to the right place. Somehow the pub/sub layer needs to know which of the templated subscriptions can actually initialize the convoy set based on the message values and which of the subscriptions simply requires the values to have already been filled. So for a parallel convoy, any subscription can activate the convoy. For a sequential convoy, only the first receive action can activate the convoy. If a message comes for the second receive before the first, it will fail routing. Again, this is just a feature of your business protocol. If you know what order the messages are required to come in, then you can have a sequential convoy. If the messages can come in any order, you might have a parallel convoy. It is really that simple.

            Sorry this isn’t the most well formatted blog, but I felt like I have been slacking (okay I have been slacking) and wanted to get you some more stuff to fill your head with. In my defense, I decided two and ½ weeks ago to do an Olympic distance triathlon and so had to try and pack training into 1 ½ weeks (especially after rolling my ankle last Monday. For those of you concerned, I finished and did pretty well for a guy who only trained for a week and a half (www.racecenter.com/hagglake).

            Adendum to my last posting on zombies. I looked into it and discussed it with a coworker of mine and I was wrong about the “virtual zombies”. The orchestration engine does some trickery and ends up causing me to create full-fledged zombies, so if a schedule is going to complete in the “Completed with discarded messages” state, the zombie wmi event will always be thrown. Sorry for the brief bit of confusion around that.

            Possible topics for next time, more convoys, when to use mappings, or my favorite, a discussion on the rising cost of rotisserie chicken in the cafeteria. Perhaps a tournament challenge to guess the price each week with winner getting the question of his/her choice answered (I am not a disgruntled employee, I can handle 25 cents, it is just a running joke between myself and some co-workers).

Thanks
Lee

Further reading: https://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdk/htm/ebiz\_prog\_orch\_iljk.asp

Comments