Towards a shared global integration model

I'm renewing my call, now over a year old, for creating a single model for integrating all open, shared services.

I'll talk about what this is, and then what benefits we get.

A Shared Global Integration Model

The idea behind a shared model is that we can take an abstract view of the way systems "should" interact.  We can create the idea of a bus and a standard approach to using it.

If we have a standard model, then we can allow a customer, say an Enterprise IT department, to purchase the parts they need in a best of breed manner.

So let's say that in Contoso, we have two systems that provide parts.  In the diagram below, Customer and Product and Marketing functions are provided by one package, while Accounting and Manufacturing and Order Management are provided by another.  They are integrated using a message bus.

The advantage of a standard approach comes when change occurs.  (Yes, Virginia, Change does occur in IT ;-)

Let's say that a small upstart Internet company creates a set of services that provides some great features in Customer management.  This is healthy competition.  (see the benefits, below).

Let's say that the CIO of Contoso agrees and wants to move the company to that SaaS product for Customer management.

That's the vision. 

Of course, we can do this without standards.  So why would we want a standard?

The benefits of a standard approach

1. Increased Innovation and Investment

We lower the economic barriers for this product to exist in the first place.  We want a small upstart company to emerge and focus on a single module.  That allows innovation to flourish.

We, as an industry, should intentionally lower the "barrier to entry" for this company.  We need to encourage innovation.  To remove these barriers, that young company should not have to create all of the modules of a system in order to compete on any one part.  They should be able to create one module, and compete on the basis of that module. 

2. Quality Transition tools and reduced transition costs

A standard approach allows the emergence of tools to help move and manage data for the transition from one system to another.  The tools don't have to come from the company that provides the functionality.  This allows both innovation and quality.  A great deal of the expense of changing packages comes from the data translation and data movements required.  Standard tools will radically reduce these costs.

3. Best of breed functionality for the business

We want our businesses to flourish.  However, these are commodity services.  Providing accounting services to a business rarely differentiates that business in its market.  On the other hand, the failure to do supporting functions well can really hurt a company.  There is no reason for the existence of an IT department that cannot do this well.  By using standards, we create a commodity market that allows IT to truly meet the needs of the business by bringing in lower cost services for non strategic functions.

4. Accelerate the Software-as-a-Service revolution

We, in Microsoft, see a huge shift emerging.  Software as a Service (SaaS) will change the way that our customers operate.  We can sit on the sidelines, like the railroad industry did in the early 1900's, as the emergence of the automobile eventually replaced their market proposition in the US and many other countries.  Or we can invest in the revolution, and give ourselves a seat at the table.  We plan to have a seat at that table. 

A shared set of service standards can radically accelerate the transition to the SaaS internet.  That is what I'd like to see happen.

A dependency on a shared information model

This movement starts with a shared information model, but not a single canonical schema or shared database.  We need to know the names of the data document types, the relationships between them, and how we will identify a single data document.  (I use the vague term "data document" intentionally, so allow me to avoid "defining myself into a corner" at this early stage.)

By having a shared information model, we can create the "thin middle" that forms the foundation for an IFaP, and middle-out architecture.

I care about this.  I believe that IT folks should lead the way, not stand by and let vendors define the models and then leave us to run around like crazy people to figure out how to integrate them.  I'd LOVE to see the "integration consulting industry" become irrelevant and unnecessary. 

It is time.

Comments

• Anonymous
  January 09, 2008
  PingBack from http://geeklectures.info/2008/01/09/towards-a-shared-global-integration-model-2/

• Anonymous
  January 09, 2008
  Interesting post. The NGOSS and OSS/J initiatives by the TeleManagement Forum show that it is actually possible (though not easy) to construct such models and I find them really helpful in my everyday job despite being one of the "integration consulting industry" :-)

• Anonymous
  January 09, 2008
  Hi Uros, In MS, we use the NGOSS and TM Forum information in our own planning and development cycles, even though we are not a Telecom.  Part of my renewed call comes from recent experience with these models. But TM Forum is focused on telecom only.  There is scant documentation of the commodity services that would form the 'sweet spot' for shared internet services and SaaS.   It's a different approach, but NGOSS proves that it is possible. --- N

• Anonymous
  January 09, 2008
  REST is not enough.  I just read Steve Vinoski's article " Serendipitous Reuse? " in Computer. 

• Anonymous
  January 09, 2008
  REST is not enough.  I just read Steve Vinoski's article " Serendipitous Reuse? "

• Anonymous
  January 10, 2008
  Isn't this what various standards organisations have tried (are still trying) to establish:

• UBL

• CCTS (UN/Ceefact) -- and many other domain specific standards such as FixML. What about things like RDF, SDO - how do they fit/ not fit here? Also - how would a shared "Integration Model" differ from a shared "Composition Model" ala SCA? I note that JJ Dubray has been calling for Microsoft to jump on board with SCA. What say you Nick? Note: I'm pretty neutral about SCA - and I'm also skeptical about these efforts to standardize all business data (have you ever read Bill Kent's classic "Data and Reality"?). But it would be interesting to get your perspective on all these existing and previous attempts to solve these problems and why you think something else is needed.

• Anonymous
  January 12, 2008
  The comment has been removed

• Anonymous
  January 22, 2008
  The comment has been removed

• Anonymous
  January 23, 2008
  Hi JJ, Thank you. I agree that evolution is needed, but I'm siding with Tim Berners-Lee on the value of standards on this one. I'm taking the long view, and if you do, then a 15-year standard-adoption-revision cycle is not only quite rational, but clearly needed.  We haven't started the wheel rolling... that's all. Neither of us have ever been the kind of person to duck from a problem because it is difficult.   I also think that we cannot evolve the programming models unless we put evolutionary pressure on them.  Nothing evolves without serious evolutionary pressure.  An effort to develop the interfaces that allow such revolutionary change would provide one impetus.   Mechanical engineers did not develop composition patterns because they were fun to think about.  They developed them because industry needed to use them.  If we work to create the demand, vendors (including my own company) will have something to drive their cycles of innovation. By calling for this process to start, we begin to create that demand.

• Anonymous
  January 23, 2008
  The comment has been removed

• Anonymous
  January 23, 2008
  Hi JJ, First off, ACORD, if I understand correctly, is similar to Rosetta.  The organization is focused mostly on the value chain and therefore B2B although you certainly CAN use the data standards within the enterprise.   This approach puts the business transaction first.  Since a business transaction, in an integrated environment, must derive from a consistent data model, and since most standards organizations do not start with a standard data model, then the business transactions produced, in most cases, are of limited value within the company.  That is because the business will communicate, between departments, using data models that reflect it's own information architecture.  Once you cross the boundaries of the wall, then the business translates their information into a business transaction. Of course, this is tedious.  Without a common data model, then every transaction has to be carefully hand-crafted, and it takes a long time to create each one.  That is why standards bodies exist... because they must. Since these bodies focus on the B2B opportunities, the areas they tackle are the areas of key value to their value proposition.  In insurance, they share claims.  In supply chain, they share shipments and bill-of-materials.  These are the stuff of the value-edge. They are NOT the stuff of the commodity center, where SaaS vendors get their biggest value story.  The commodity center is the place where commodity transctions matter.  Things like "human resource" and "travel expense request" are the interesting messages for the commodity hub at the center, because they are the transactions that should be outsourced. So I'm not calling for some new industry consortium to create yet-another-data-communication standard.  I'm talking about a base information model and base integration model, and system descriptions for 100 different business application categories, complete with details of how each and every one will integrate with it's partners.  It's a shared architecture, one that any enterprise can use to outsource core (commodity) transactions. That represents a level of pre-integration that cannot be done with the standards as they exist today. And I'm witholding my opinion on SCA  vs. .Net.  I will say this: a packaging model will solve some problems, but it won't solve this one.

• Anonymous
  January 24, 2008
  The comment has been removed

• Anonymous
  January 24, 2008
  Hi JJ, I am familiar with both OAGIS and SCA.  The fact that I'm not responding to your questioning on SCA does not imply that I fail to understand.   OAGIS, if you look closely, provides many models for integrating systems.  You are encouraged to "pick the model that is the closest to what you need" and then use the transactions according to that model. I like the envelope from OAGIS.  I like the verb-noun model.  I like the seperation of data from command (a pattern also used in REST, BTW).  I am not going to suggest otherwise. However, if Joe picks one model for Order to Cash for his business, and Mary picks another model for Order to Cash for her business, then a service provider that wants to write a service that both Joe and Mary can purchase has to support both models. And therein lay the rub.   OAGIS is a transactional standard.  It is a nice one.  But it is not an architectural model. What I am talking about will define the boundaries of the system... say the personnel system and the ERP system.  One standard interface.  Async, of course (especially since you like SCA :-). One model to integrate them.  One definition of the boundaries.  Clear, consistent, and standard service names.   There will be a HUGE amount of flexibility WITHIN the system boundary, but no flexibility at all on the system boundary.  It is defined.  It is done.  It is standard. If any of you think this is draconian, dear reader, consider this: the electrical system in your house is exactly this.  Completely standard, and yet you can plug in that new 42-inch flat screen TV into the same system that was designed almost 100 years ago.   Standardization of the system did not prevent or obstruct innovation at all.  On the contrary, it empowered it.   So, JJ, I'm not talking about a transaction model like OAGIS, and I'm not talking about an async callback to proxy mechanism like SCA.  They are interesting and useful, but insufficient by themselves to meet the needs of the next generation of software.   They are small, evolutionary innovations. I'm calling for a large, revolutionary, disruptive shift. Nothing less will do.

• Anonymous
  January 25, 2008
  The comment has been removed

• Anonymous
  January 26, 2008
  The comment has been removed

• Anonymous
  January 28, 2008
  When I opened my call for a Shared Global Integration Model , I expected some folks to say "we don't

• Anonymous
  January 29, 2008
  Harry "Devhawk" Pierson, whom I'm glad to count among my friends, sent me an e-mail last week. 

• Anonymous
  February 04, 2008
  I've been on a roll lately, calling for the creating of a standardized approach to the partitioning of

• Anonymous
  February 04, 2008
  I've been on a roll lately, calling for the creating of a standardized approach to the partitioning