Enterprise SOA needs a Federated Evolutionary Modeling Environment

I've been thinking a lot lately about the gap between "what we have" and "what we need" in the Enterprise SOA space.  I think I have a need that is not yet filled by software.  (that I'm aware of).

I put up a post back in June about the difficulty in creating a common information model in a large enterprise, especially one with a federated environment like ours.  (CISR coordination and diversification models).  The feedback I got back was telling.  The majority of respondants told me what I suspected: developing an enterprise-wide common information model was so difficult that many folks considered it unfeasable.  (Actual words included things like "utopian" and "big design up front."  I'll stick to "unfeasable").

That said, I have also stated in public that I believe, firmly, that SOA at the enterprise level requires some levels of agreement on the way that information is understood and coordinated.  Each business unit can own information that is specific to that unit, but in the areas of coordination, where there is value, the business needs to be able to communicate.

So, we have something that is hard to do (build a CIM and keep it up to date).  That something is useful (to get the benefits of Enterprise SOA).  So, why not take the software approach?  After all, I do work at Microsoft.

What business scenario would this tool need to support?

Basically: each business unit would submit their information model to a common repository.  The submitters are architects, and they MUST align the models in some way, even if it is just to show that there are differences.  Services are posted to the same repository, and must be lined up under specific information models.  In order to consume a service, the business has to agree to the information model.  Multiple competing models are allowed to co-exist.  What do we get?  An economic model of self-governance that produces an optimum information model: one where agreement is reached only where agreement makes financial sense.

Specific capabilities:

• A business unit architect can consume part of an information model without consuming the entire thing.
   
• A business unit architect can assert part of an information model without proposing the entire thing.
   
• A business unit architect can offer services tied to their information model.
   
• A business unit architect can assert that a portion of an information model is "standard" for their use
   
• A developer, writing software for that business unit, can easily find and adopt their information model.
   
• A project team can provide a report to a governance committee proving that they are conformant to local standards
   
• An enterprise architect can run reports to isolate "points of difference" between business unit information models.
   
• A system designer, intent upon consuming services from another business unit, can begin a workflow to insure that the consuming business unit agrees to the information model of the presenting business unit. (an "information adoption workflow").
   
• The workflow needed for a consuming business unit to agree can be custom-tailored to the organization.
   
• Organizations that present a service have an automatic measurement mechanism built in allowing them to "charge back" the businesses that consume the service.  Various financial models must be supported (one time fee, pay per use, annual licenses).  This provides economic incentive for sharing of code, as well as the incentive to create services that align to commonly needed information models.
   
• Built in support for the versioning of information models over time (including both past and future versions) allows the business to change their minds, and even chart their course.

That's what my gut tells me.  This has some pretty interesting effects:

1. Information architects have a clearly defined and critical role at the earliest stages of a project: get consensus on information model changes needed to allow the consumption of existing, lower cost services.
    
2. Economics will drive good behavior.  No need for an Enterprise Architect to "design" good behavior. 
    
3. Less emotion.  There will not be consensus on everything, and this model doesn't require it.  Consensus will be reached surprisingly easily on some key areas, and it will happen without any architect looking to make it happen.  This helps remove politics from the picture as well.
    
4. It is easier to adopt existing code than build new code if services, offered by other groups, aligned with the information standards of the consuming business, are already clearly identifiied. 

We may be closer than we think.  With bits from various MS products, and with Oslo coming, this vision is getting closer to reality.  It's an end-to-end idea. 

Something to consider.  Comments?

Comments

• Anonymous
  August 05, 2008
  Nick, Good post... but I'm wondering if you're overlooking the obvious. By the way, I've put a post up that got a few comments about how "challenging it would be to achieve [what I suggested] in a large organization." (http://bit.ly/YtX2c) My point is... large organizations already have solved this problem. Whether HIPAA or FIX or SWIFT, each of this is a multi-organization data standard. Each achieved without dumping stuff in a registry. Let's look at what the differences are:

1. The organizations I mentioned are specifically motivated to deliver a common data model.
2. There is a clear business driver (more rapid communication that can be more easily understood between organizations). I think the effort should be made to motivate properly and address the business drivers, rather than on technology (registry) adoption. By the way, I admit, these data standards only address the data model, and not how it's used between organizations, which is something you mention above as well, that I'm not addressing. david

• Anonymous
  August 05, 2008
  The comment has been removed
• Anonymous
  September 02, 2008
  Nick,

1. Although we can implement SOA without CIM (or canonical data model), we really need it for EDA. So your post has sense certainly...
2. What you model CIM (or CDM) in? I do not see any problem with splitting model to parts and sharing them with existing tools (if you have CASE with shared repository, most of your requirements can be implemented easily). I do not see any problem with functions as workflow (it can be implemented as add-on to CASE if needed).
3. But I see the problem with "adopting" local standards. What I tried to tell in my blog entry (http://it.toolbox.com/blogs/system-integration-theory/canonical-data-model-using-industry-standard-data-models-26635) was, that existing schema languages are not ready for "adoption" of industry standard model. Support for evolutionary modeling (from industry standard models to local standard models and from local standard models to business unit models) would be great... You work in Microsoft, you could solve it somehow :)

• Anonymous
  September 03, 2008
  Hi PR, (I assume this is Peter Rajsky) Yes, I work in MSFT.  It's an interesting problem.  I don't know if I'll get the chance to solve it, but if I do, then you'll know where I first documented the idea. --- Nick