Recurring Architectural Capabilities

More fluffy draft on yet another fluffy topic...

Again, feedback encouraged.

--

Recurring Architectural Capabilities

As we saw earlier, the SOA architectural model is fractal. This means that a service can be used to Expose IT assets (such as a Line of Business system), be Composed into workflows or Business Processes (each of which may also be exposed as a service) and be Consumed by end users, systems or other services. SOA is a fractal, not layered model. While the Abstract SOA Reference Model provides a holistic view of several important SOA concepts, the Expose / Compose / Consume portions of the model should not be interpreted as layers (despite their apparent appearance in the model). Designing a SOA as a set of well-defined tiers (or layers) will constrain the value and flexibility of your services, resulting in dependencies across unrelated components. This is why the Expose / Compose / Consume portions of the model can be thought of as independent architectural initiatives referred to as a Service Implementation Architecture (Expose), a Service Integration Architecture (Compose) and an Application Architecture (Consume). While each of these architectures are designed to be independent of one another, they share a set of five common capabilities

Figure 7: Recurring Architectural Capabilities

Communications

Communications focuses on how messaging is accomplished between senders and receivers. There are a broad array of options and patterns available – from pub/sub and asynchronous to message and service interaction patterns. Services provide an evolutionary approach to building distributed software that facilitates loosely coupled integration and resilience to change. The advent of the WS-* Web Services architecture has made service-oriented software development feasible by virtue of mainstream development tools support and broad industry interoperability. While most frequently implemented using industry standard Web services, service orientation is independent of technology and architectural patterns and can be used to connect with legacy systems as well. Messaging and Services are not a new approach to software design – many of the notions behind these concepts have been around for years. A service is generally implemented as a coarse-grained, discoverable software entity that exists as a single instance and interacts with applications and other services through a loosely coupled (often asynchronous), message-based communication model. Messages tend to be based upon an agreed-upon set of semantics (such as an industry-standard Purchase Order) and serialized using an interoperable, extensible syntax (usually XML, although alternate approaches like JSON, RNC and ASN1 are sometimes used).

Workflow and Process

Workflow and Process are pervasive across multiple layers of an integration architecture – from formal orchestration of processes to flexible ad hoc systems and collaborative workflow across teams. Since business processes are dynamic and evolve with the organization, the workflow that models these processes must be equally adaptable. In addition, effective workflow involves not only process modeling, but also monitoring and analytics in order to respond to exceptions and optimize the workflow system over time.

Data

The lynchpin to success in many integration architectures is the ability to provide Data management. The need to deliver a shared view across disparate, often duplicate sources of data is more important than ever, as businesses strive to achieve a 360-degree view of organizational information. Entity aggregation, master data management, and the ability to make data useful via analytics and mining are crucial elements of integration architecture.

User Experience

Successful integration architectures depend upon both service delivery and the ability to consume services in a rich and meaningful way. Service consumption needs to be contextual, mapping to the natural workflow of employees, customers, and partners. To that end, an integrated User Experience spanning smart clients, rich clients, lightweight Web applications, and mobile devices enables service consumption by the broadest possible audience.

Identity

To support integrated user experiences, customers require the ability to manage the identity lifecycle – providing integrated Single Sign-On (SSO), access management, directory services, and federated trust across heterogeneous systems. Today, many solutions are built using fragmented technologies for authentication and authorization. In the new application model, access decisions and entitlements need to be made at multiple layers and tiers, in which a federated Identity across trust boundaries becomes a key requirement.

Management

During the lifetime of a service, the service most probably changes in different perspectives as listed below. As a result, one service will probably have to be available in several versions.

• Difference in interface (e.g. extended interface, but same business object)
• Difference in semantics with same interface (business objects changed)
• Difference in QoS, e.g. slower but cheaper or high-available but more expensive

Service management encompasses many capabilities, some of which are listed below (we will discuss Management in greater detail in Chapter ?? ).

• A comprehensive solution for change and configuration management, enabling organizations to provide relevant software and service updates to users quickly and cost-effectively.
• Reducing the complexity associated with managing the IT infrastructure environment with a focus on lowering the cost of operations.
• Centralized backup services capturing changed files to disk. Centralized backup should enable rapid, reliable recovery from disk while providing end-user recovery without IT intervention.
• Pre-deployment capacity planning coupled with best-practice guidance and hardware-specific knowledge to help information technology professional make low-risk architectural decisions.
• Data warehouse and reporting to help IT better support corporate decision making, improve the quality of service provided, and better administer resources through improved reporting capabilities and management data integration from a broad variety of resources.

Supporting the Common Architectural Capabilities

The five architectural capabilities discussed above are supported by the Microsoft SOA Platform. Support for the common architectural capabilities is discusses in greater detail in Chapter 2.

Figure 8: SOA Capabilities on the Microsoft Platform

Common Architectural Capabilities and the Abstract SOA Model

We can also think of these five common architectural capabilities as set of perspectives for viewing and understanding the Abstract SOA Model. The five architectural capabilities serves as a set of lenses to help us view and better understand the challenges associated with Exposing existing IT investments as services, Composing the services into business processes and Consumingthese processes across the organization.

Expose

Service Enablement

Expose focuses on how we design and expose our services. We will most likely start out by enabling our IT investments to be exposed as web services.

As our organization matures we will start adding new services, most likely as proxies for other resources within the organization.

One of the hardest parts of service implementation is deciding where to begin. The are a variety of choices here and there is no single recommendation that will work for everyone. Motion is a methodology that provides some guidance for identifying business capabilities that could be exposes as services.

What are some best practices that one should follow when exposing IT investments as services?

• Think big– but start small
  • Show value at every step along the way – not build it and they will come
• Middle-out – not Top-down or bottom-up
  • Be pragmatic
• Vertical-slice – not build it and they will come
  • Risk-mitigation
• Demonstrate value in rapid iterations – not waterfall
• Successful customers ‘snowball’

The recurring architectural capabilities provide us with a set of considerations when exposing IT investments as services. Let’s take a quick look at some of the considerations associated with each capability for service exposure (this is by no means a complete list):

Communications

Determining what to expose and how - Avoid falling into the granularity trap – focus on meeting your business requirements

Service Operation Contracts

Message and Data Contracts

Configuration, behaviors and control

SLAs

Governance

Versioning

More…

Workflow and Process

Coordinator services for distributed, long-running processes

Tracking services capable of logging specific events within a workflow

Compensation services

Data

Entity services

Entity Aggregation services: acts as a single point to access information that may exist in multiple systems. An Entity Aggregation service has the following responsibilities:

• • Acts as a unified source of entities.
  • Provides a holistic view of an entity.
  • Provides a holistic view of the entity model—entities and their relationships with other entities
  • Provides location transparency—consumers of the Entity Aggregation layer do not need to know who owns the information.
  • Enforces business rules that determine the segments of entities retrieved in a given context.
  • Determines the system of record for each data element constituting an entity.
  • Enriches the combined data model across systems—the whole being better than the sum of its parts.
  • Entity factoring

MDM – exposing data across corporate or departmental boundaries.

Canonical schemas imply that all the services share the same schema, which need not be the case

User Experience

Specialized services for supporting user interfaces (caching resources, communications between UI and services, etc). Service wrappers provide coarse-grained interfaces for user app consumption, lightweight mash-ups, etc.

Identity and Access

Identity Management

Impersonation and Delegation services

Trusted Subsystem - A trusted subsystem model implies that services are trusted to perform specific tasks, such as processing customer orders.

Authentication (Kerberos, SSL)

Role-based access control (RBAC)

Create/revoke trust relationships

Services need to make authorization decisions, such as approving an order submission before performing the business transaction.

The service must know the identity of the end user submitting the order.

Need to flow the identity of the end user is an inherent property of the delegation model, it is not so for the trusted subsystem model and special efforts must be made to include this feature.

To support the notion of trust as defined by the model, the services must at least be able to:

1. Authenticate / verify identity of upstream / downstream services
2. Decide if the service is a trusted subsystem for specific functions (including propagating identity claims)
3. Protect the integrity of the data being communicated between trusted subsystem and services.

Besides application data, application plumbing data, such as the identity claims of the original user, must also be protected so that no man-in-the-middle can modify the identity information that is in transit.

Compose

Service Composition

Compose focuses on how we can combine or aggregate granular services into more complex processes. We will most likely start by using services that expose our existing IT investments.

Service composition results in a new service instance that the rest of the organization can make use of. The composition provides capabilities such as correlated asynchronous service invocation, long running processes and other capabilities for orchestrating autonomous services.

The recurring architectural capabilities provide us with a set of considerations when composing granular services into complex processes. Let’s take a quick look at some of the considerations associated with each capability for service composition (this is by no means a complete list):

Communications

Service interaction patterns

Exposing orchestrations as services

Asynchronous service invocation patterns

Workflow and Process

Transactions

High frequency of change

Business Rules

Service Orchestration

Service Interaction Patterns (SIPs)

Process Externalization

Long Running Processes

Auditing and analytics

Data

Tracking the state of a given workflow instance

Data transformation (ETL)

Reliable message processing and storage

Replication

Synchronization

Metadata repository and Management

Instance reconciliation

Schema reconciliation

Document Replication

Syndication/Aggregation

User Experience

Composite applications (OBAs)

Human workflows (MOSS)

Orchestrations initiate human workflows via SharePoint adapter

Pageflows

Identity and Access

Impersonation and Delegation

Provisioning

Identity Repository synchronization

Approval workflows

Consume

User Experience

Consume focuses on how services and orchestrated processes (which may be exposed as services) are consumed by other services, applications and end-users. Any resource capable of interacting with services can be referred to as a “consumer”. Consumers may appear across the organization in several possible forms:

• Lightweight, browser-based applications
• Rich Internet applications (RIA) are browser-based applications that can address and cache local and remote resources
• Configurable, portal-based user experiences
• Applications that are installed on the local machine (such as a custom Windows application)
• Corporate business applications with solution-specific extensions (such as Microsoft Office with context-aware activity panes)
• Applications designed for mobile devices
• Services may act as consumers of other services

Recall that the SOA model is fractal – services may be consumed by other services and service compositions may be exposed as new services. In the last couple of years a “new breed” of consumers has emerged, enabling consumers to be aggregated and consumed by additional consumers. This “new breed” of consumers is usually referred to as a “mashup”. A mashup is a set of services, websites or applications that combine content from multiple resources into a new integrated user experience. Content used by mashups is typically sourced from a third party (such as a service or website) via a public interface or API. Alternative methods of sourcing content for mashups include newsfeeds and JavaScript constructs (JSON).

The recurring architectural capabilities provide us with a set of considerations for User Experience. Let’s take a quick look at some of the considerations associated with each capability for User Experience (this is by no means a complete list):

Communications

Forms-based service consumption

Web parts

Service Registry – check in / check out / search

AJAX, REST

Workflow and Process

Human workflows (MOSS)

Event brokering (CAB)

Page flows

Data

Entities (OBA Business Data Catalog)

Single view of the customer problem

JSON

User Experience

Composite applications (OBAs)

Personalization, user profiles

Portals

Business Intelligence

Reporting

Content aggregation

Declarative UX

Identity and Access

Single Sign-On (password synchronization)

User identification

Role-based access (RBAC)

Directory Services

Password management

Privacy (firewalls, encryption)

Compliance

Summary

In this chapter we provided some useful analogies for understanding the fractal nature of SOA. Services are the fundamental building blocks of SOA, although services do not necessarily need to be web services. Ideally these services should follow the four service design tenets which describe a set of best practices for service scopes, dependencies, communications and policy-based configuration. While these tenets focus upon service design, it is important to realize that services alone are not necessarily solution architecture – Microsoft uses an abstract reference model to describe the various aspects of SOA. The abstract SOA reference model provides three fundamental concepts to help most organizations understand the role that services can play within their solution architectures:

• Expose focuses on how existing IT investments are exposed as a set of broad, standards-based services, enabling these investments to be available to a broader set of consumers. A Service Implementation Architecturedescribeshow services are developed, deployed and managed.
• Compose focuses oncombining services into applications or cross-functional business processes. A Service Integration Architecture describesa set of capabilities for composing services and other components into larger constructs such as business processes.
• Consume focuses on delivering new applications that enable increased productivity and enhanced insight into business performance. A Service Oriented Application Architecture describes how “composed services” are made available for consumption through as business processes, new services or new end-user applications.

Each aspect of the Expose / Compose / Consume abstract reference model encompasses a set of five recurring architectural capabilities: Communications, Workflow and Processes, Data, User Experience and Identity. The five architectural capabilities serves as a set of views to better understand the challenges associated with Exposing existing IT investments as services, Composing the services into business processes and Consumingthese processes across the organization.

Comments

• Anonymous
  April 12, 2007
  More fluffy draft on yet another fluffy topic... Again, feedback encouraged. -- Recurring Architectural

• Anonymous
  April 12, 2007
  The comment has been removed

• Anonymous
  April 25, 2007
  Hi there, in figure 8, you show MSMQ falling under the Comm. section, and SSB falling under the Data section. Aren't they somewhat competing technologies?

• Anonymous
  June 01, 2008
  The comment has been removed