Design for Flexibility

First of all, let’s level set on the definition of system Flexibility. I quite like M. Barbacci’s definition which is “Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.”

The beauty of this system quality attribute is that it is a key factor in providing Agility ("The ability of a system to be both flexible and undergo change rapidly" --MIT). In fact, I’ll blog some ideas I’ve been thinking as a formula for Agility because I feel Flexibility doesn’t seem to get the credit it deserves here…anyway, for arguments sake, let’s just assume that Flexibility is an important system quality attribute.

The problem I’d like to raise for discussion is how to help system designers build systems that are optimized for Flexibility. One approach I've been thinking about to help system designers build systems that are optimized for system Flexibility is to create an Architecture Principle which includes direction and explanation so that it can easily be consumed by system designers. I’d like to throw one out to this Architect Community to see what you all think.

The approach I’ve taken is

1.      Identify well-known patterns that optimize for Flexibility in high and low level system abstraction levels.

2.      Identify common scenarios in my IT environment and use them to explain what to do in those scenarios to optimize for Flexibility

3.      Identify Flexibility anti-patterns, or patterns which degrade system Flexibility.

Using this approach this, I’ve created a document which contains the content of this design principle. Below is a first go at this principle:

Design for Flexibility

Description: Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.

Rationale:

Optimizing for system Agility is a primary goal for enterprise systems with the purpose of enabling rapid delivery of enterprise applications to the business. There are several system quality attributes which are used to calculate system Agility such as Reusability, Interoperability and Flexibility. Within Microsoft IT’s enterprise application environment, system Flexibility is of particular importance as it is a key factor contributing to an agile architecture. For this reason, attention is paid to ensure guidance is provided to system designers to ensure systems are optimized for Flexibility.

Implications:

If systems are designed to optimize system Flexibility, the expected benefits are:

· Enabling business process changes with minimum effect on technology platforms.

· Enabling technology platform changes with minimum effect on business processes.

· Enabling Service Oriented Architecture (SOA).

· Enabling modularized systems library for rapid system reuse.

· Improving system Maintainability.

A note about when and where to optimize for Flexibility: This sections includes a few scenarios commonly observed in Microsoft IT to help guide system designers where and how to optimize for Flexibility.

1. System Integration. There are three primary Interface Types to send and receive data between systems and they are; Real-time Integration, Near Real-time Integration and Batch Integration. Each of these scenarios are explained in the sections below.

a. Real-time Integration

                                          i. The table below describes the scenario when Real-time Integration is often the best choice:

Term	Value
Transaction Volume – Traffic	Greater than 1 Peak Transactions Per Second
Transaction Volume – Message Size	Less than 10Mb Peak Message Size per Transaction
Interface Response Time	Less than 5 seconds
Latency Tolerance	Less than 5 seconds
Data State Staleness	Less than 60 seconds

                                         ii. The table below lists patterns and styles which optimize systems for Flexibility for Real-time Integration scenarios. See Appendix A - Pattern Reference for details on these patterns.

Name

Adapter

Chain of Responsibility

Command

Data Transfer Object

Entity Aggregation

Event Message

Event-Driven Consumer

Façade

Functional Integration

Gateway

Message Broker

Message Bus

Message Endpoint

Messaging

Messaging Gateway

Observer

Pub/Sub

Remote Façade

Service Layer

Service-oriented Integration

b. Near Real-time Integration

                                          i. The table below describes the scenario when Real-time ETL is often the best choice:

Term	Value
Transaction Volume – Traffic	Less than 1 Transactions Per Second at Peak period and greater than 1 Transaction per 30 minutes at Trough period
Transaction Volume – Message Size	Greater than 10Mb and less than 30 Mb Peak Message Size per Transaction
Interface Response Time	Greater than 5 seconds and less than 3 minutes
Latency Tolerance	Greater than 5 seconds and less than 30 minutes
Data State Staleness	Less than 30 minutes

                                          i. The table below lists patterns and styles which optimize systems for Flexibility in Near Real-time Integration scenarios. See Appendix A - Pattern Reference for details on these patterns.

Name

Adapter

Chain of Responsibility

Command

Data Transfer Object

Entity Aggregation

Event Message

Event-Driven Consumer

Façade

Functional Integration

Gateway

Message Broker

Message Bus

Message Endpoint

Messaging

Messaging Gateway

Observer

Pub/Sub

Remote Façade

Service Layer

Service-oriented Integration

c. Batch Integration (aka ETL )

                                          i. The table below describes the scenario when Batch Integration via ETL is often the best choice:

Term	Value
Transaction Volume – Traffic	Less than 10 Transactions per day at Peak period
Transaction Volume – Message Size	Greater than 30 Mb Peak Message Size per Transaction
Interface Response Time	Greater than 3 minutes
Latency Tolerance	Greater than 30 minutes
Data State Staleness	Greater than 30 minutes

                                         ii. To optimize Flexibility in Batch Integration scenarios,

1.  Use the following Object-oriented Design patterns; Façade and Adapter even within SQL Stored Procedures. See Appendix A - Pattern Reference for details on these patterns.

Name

Adapter

Façade

2. Follow guidelines to implement the concepts of encapsulation and ‘separation of concerns’.

d. SAP Web Services. This scenario is when an application requires integration to an SAP BAPI which is wrapped with a Web Service application. To optimize for Flexibility, the Web Services which directly wraps the SAP BAPI should implement the following Object-oriented Design patterns; Façade, Canonical Data Model and Adapter. See Appendix A - Pattern Reference for details on these patterns.

2. Configuration Management scenarios. Systems which have been designed to optimize for Flexibility take advantage of configuration mechanisms allowing for changes to the behavior of an application via changes in a system configuration file.

a. Types of Configuration Management changes that optimize for Flexibility are

                                          i. Configurable Logging sources and log message types

                                         ii. Configurable modules for in-process interception – Note that the Chain of Command Object-oriented Design pattern should be used in these situations.

Measure:

· é Green – The use of Architecture patterns which optimize Flexibility are used at all architecture and design abstraction levels as well as an implementation of Configuration Management for all major non-functional and functional sub-systems and system components.

o Examples of proven Architecture Styles and Architecture and Design patterns which optimize system Flexibility are in the table below. See Appendix A - Pattern Reference for details on these patterns.

Name

Adapter

Chain of Responsibility

Command

Data Transfer Object

Entity Aggregation

Event Message

Event-Driven Consumer

Façade

Functional Integration

Gateway

Message Broker

Message Bus

Message Endpoint

Messaging

Messaging Gateway

Observer

Pub/Sub

Remote Façade

Service Layer

Service-oriented Integration

o Types of Configuration Management changes that optimize for Flexibility are

§ Configurable Logging sources and log message types

§ Configurable modules for use process interception – Note that the Chain of Command Object-oriented Design pattern should be used in these situations.

Separation

· èYellow – Architecture patterns which optimize Flexibility at the design abstraction level. Examples of such design-level patterns which optimize system Flexibility are:

Name

Adapter

Chain of Responsibility

Command

Data Transfer Object

Entity Aggregation

Event Message

Event-Driven Consumer

Façade

Message Endpoint

Observer

Remote Façade

Note: See Appendix A - Pattern Reference for details on these patterns.

· êRed – System implementations which:

o Do not implement any of the styles and patterns listed in Appendix A - Pattern Reference with a Direct or Indirect value in the “Relevance to Flexibility” column.

o Are using patterns such as

§ Shared Database

§ Remote Procedure Invocation

o Do not follow guidelines for Encapsulation and Separation of Concerns

o Do not have configurable sub-systems or application components.

o Implement a Batch Integration system integration type for a Real-time or Near real-time scenario

---

1.1Appendix A - Pattern Reference

Name	Relevance to Flexibility	Type	Reference
Remote Procedure Invocation	Anti-pattern	Style	https://www.enterpriseintegrationpatterns.com/EncapsulatedSynchronousIntegration.html
Shared Database	Anti-pattern	Style	https://www.enterpriseintegrationpatterns.com/SharedDataBaseIntegration.html
Adapter	Direct	Design	https://www.dofactory.com/Patterns/PatternAdapter.aspx
Event Message	Direct	Design	https://www.enterpriseintegrationpatterns.com/EventMessage.html
Façade	Direct	Design	https://www.dofactory.com/Patterns/PatternFacade.aspx
Functional Integration	Direct	Architecture	https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archfunctionalintegration.asp
Messaging	Direct	Style	https://www.enterpriseintegrationpatterns.com/Messaging.html
Service Layer	Direct	Architecture	https://www.martinfowler.com/eaaCatalog/serviceLayer.html
Service-oriented Integration	Direct	Architecture	https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archserviceorientedintegration.asp
Chain of Responsibility	Indirect	Design	https://www.dofactory.com/Patterns/PatternChain.aspx
Command	Indirect	Design	https://www.dofactory.com/Patterns/PatternCommand.aspx
Data Transfer Object	Indirect	Design	https://www.martinfowler.com/eaaCatalog/dataTransferObject.html
Entity Aggregation	Indirect	Architecture	https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archentityaggregation.asp
Event-Driven Consumer	Indirect	Architecture	https://www.enterpriseintegrationpatterns.com/EventDrivenConsumer.html
Gateway	Indirect	Architecture	¡https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/desgateway.asp ¡https://www.martinfowler.com/eaaCatalog/gateway.html
Message Broker	Indirect	Architecture	¡https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebroker.asp ¡https://www.enterpriseintegrationpatterns.com/MessageBroker.html
Message Bus	Indirect	Architecture	https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebus.asp
Message Endpoint	Indirect	Architecture	https://www.enterpriseintegrationpatterns.com/MessageEndpoint.html
Messaging Gateway	Indirect	Architecture	https://www.enterpriseintegrationpatterns.com/MessagingGateway.html
Observer	Indirect	Design	¡https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpatterns/html/DesObserver.asp ¡https://www.dofactory.com/Patterns/PatternObserver.aspx
Pub/Sub	Indirect	Architecture	¡https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/despublishsubscribe.asp ¡https://www.enterpriseintegrationpatterns.com/PublishSubscribeChannel.html
Remote Façade	Indirect	Design	https://www.martinfowler.com/eaaCatalog/remoteFacade.html

Appendix A Glossary

Term	Definition
Agility	Ability of a system to be both flexible and undergo change rapidly.
Architecture Pattern	An architectural pattern expresses a fundamental structural organization schema for software systems. It provides a set of predefined subsystems, specifies their responsibilities, and includes rules and guidelines for organizing the relationships between them.
Architecture Style	An architectural approach to solving a common problem.
Business Capability	The unit describing the combination of people, business process and technology and the policy around its management
Component	Defined as executable/source code, practices, patterns, services as prescribed by Technical Architecture team.
Data State Staleness	The difference between the master state, from which the cached state was created, and the current version of the cached state.
Design Pattern	A design pattern provides a scheme for refining the subsystems or components of a software system, or the relationships between them. It describes a commonly recurring structure of communicating components that solves a general design problem within a particular context.
Encapsulation	Encapsulation conceals the exact details of how a particular class works from objects that use its code or send messages to it. So, for example, the Dogs class has a bark() method. The code for the bark() method defines exactly how a bark happens (e.g., by inhaling() and then exhaling(), at a particular pitch and volume). Timmy, Lassie's friend, however, does not need to know exactly how she barks. Encapsulation is achieved by specifying which classes may use the members of an object. The result is that each object exposes to any class a certain interface — those members accessible to that class. For example, an interface can ensure that puppies can only be added to an object of the class Dog by code in that class. Members are often specified as public, protected and private, determining whether they are available to all classes, sub-classes or only the defining class. Some languages go further: Java uses the protected keyword to restrict access also to classes in the same package, C# and VB.NET reserve some members to classes in the same assembly using keywords internal (C#) or Friend (VB.NET), and Eiffel allows one to specify which classes may access any member
Enterprise Solution	Any system that participates in an end to end business process that spans beyond the organization. The solution is SAP, Siebel, Clarify,/Metropolis, *ebis, or a solution that masters a key data entity such as Organization, Partner, Customer, Solution Opportunity, Campaign, Product or Order (among others).
Extract Transform and Load (ETL)	Extract Transform and Load is an approach to create nearly-identical copies of the data and to manage the integrity of the copies if they can be updated at both the source and target within a replication interval. · https://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpatterns/html/ArcDataReplication.asp
Flexibility	Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.
Implementation Pattern	An implementation pattern is a low-level pattern specific to a particular platform. An implementation pattern describes how to implement particular aspects of components or the relationships between them, using the features of a given platform.
Intentional Integration	Defining a public data and service contract to the process and data owned by a system. Should not be consumer driven decision
Interface Response Time	Amount of time a consumer application is expected to wait for a response from the provider application.
Interoperability	the ability of two or more systems or components to exchange information and to use the information that has been exchanged.
Latency Tolerance	Delay between updates between the Source Application and the Destination Application. Some forms of data integration imply a delay between updates to the data that is used by multiple applications. For example, in theData Replicationpattern, the data is extracted from the source system, and it is transported over a network. The data might then be modified, and then it is inserted in a target database. This delay means that one system may have access to data that is more up to date than another system
Maintainability	Maintainability is: · The aptitude of a system to undergo repair and evolution. · The ease with which a software system or component can be modified to correct faults, improve performance or other attributes, or adapt to a changed environment. (2) The ease with which a hardware system or component can be retained in, or restored to, a state in which it can perform its required functions.
Near Real-time Integration	General term for describing asynchronous inter-system communication typically an implementation of the Message Bus Integration Style. · https://www.enterpriseintegrationpatterns.com/ComposedMessagingMSMQ.html · https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebus.asp
Performance	Performance is the responsiveness of the system – the time required to respond to stimuli (events) or the number of events processed in some interval of time. Performance qualities are often expressed by the number of transactions per unit time or by the amount of time it takes to complete a transaction with the system.
Process	A process is the business process that an application may be involved in executing or automating
Real-time Integration	General term for describing synchronous inter-system communication typically an implementation of the Request-Reply pattern.Note that Real-Time Integration can also implement asynchronous communication but in a fashion to scale synchronous communication.
Relevance to Flexibility	§ Direct. The pattern/style directly contributes to optimizing system Flexibility. § Indirect. The pattern/style indirectly contributes to optimizing system Flexibility. § Anti-pattern. The use of the pattern adversely affects system Flexibility.
Reliability	Reliability is the ability of the system to keep operating over time. Reliability is usually measured by mean time to failure.
Reusability	Reusability is the degree to which a software module or other work product can be used in more than one computing program or software system. This is typically in the form reusing software that is a encapsulated unit of functionality.
Role	A collection of responsibilities and motivations, to which authorizations are attached.
Scalability	Scalability is the ability to maintain or improve performance while system demand increases.
Security	Security is a measure of the system’s ability to resist unauthorized attempts at usage and denial of service while still providing its services to legitimate users. Security is categorized in terms of the types of threats that might be made to the system.
Separation of Concerns	Separation of concerns is the process of breaking a program into distinct features that overlap in functionality as little as possible. A concern is any piece of interest or focus in a program. Typically, concerns are synonymous with features or behaviors
Supportability	Supportability is the ease with which a software system is operationally maintained.
Testability	Testability is the degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met.
Transaction Volume – Peak Message Size	The largest message size expressed in bytes for a single transaction.
Transaction Volume – Peak Traffic	The largest number of transactions expected to be processed expressed in ‘transactions per second’.
Type	§ Architecture. See Architecture Pattern. § Design. See Design Pattern. § Style. See Architecture Style.
Usability	Usability is: § The measure of a user’s ability to utilize a system effectively. § The ease with which a user can learn to operate, prepare inputs for, and interpret outputs of a system or component. § A measure of how well users can take advantage of some system functionality. Usability is different from utility, a measure of whether that functionality does what is needed.

 

What do you think?

How do you provide Architecture/Design guidance to optimize for system Flexibility?

Comments

• Anonymous
  October 20, 2006
  What is ETL?

• Anonymous
  November 14, 2006
  Extract, Transform and Load.

• Anonymous
  July 19, 2007
  In a previous blog , I posted some considerations to assist analyzing a business for opportunities to

• Anonymous
  September 06, 2007
  Months ago I published a blog about how to implement system quality attributes ( found here ). In that

• Anonymous
  July 01, 2008
  Welcome to the July 1, 2008 edition of Carnival of Enterprise Architecture (Issue #10). Business Process Management Stephan Grindley presents Enterprise Asset Management Software - a Great Way to Manage posted at Asset Management Articles. Bozidar Spirovski