Can progress be standardized?
There’s a fundamental challenge that many standards maintenance groups and standards implementers deal with from time to time: how can standards evolve, and implementers innovate, while maintaining the sort of reliable and well-documented interoperability that standardization enables?
Standards maintenance processes add new functionality slowly and carefully, but the products that implement those standards may progress much more rapidly at times, to address new needs in the marketplace, new technologies, and ever-changing market dynamics. Is it possible to “loosely couple” a standard and its implementations in a way that allows implementers to deliver innovations while remaining interoperable and conformant to the standard?
I recently had the opportunity to work with a few of my Microsoft colleagues to put together a whitepaper (attached below) that touches on some of these issues. The paper was delivered at an interoperability event in Beijing earlier this month, and it provides an overview of MCE (Markup Compatibility and Extensibility, Part 3 of ISO/IEC 29500) and how it has been used to enable interoperability between various versions of the standard and its implementations.
There are several other strategies that have also been used to address this challenge in document formats:
- A standard may allow foreign markup to appear inline. This gives implementers the freedom to innovate in any way they’d like, but it also means that documents can’t be validated against the published schemas in the standard. Removing the foreign markup is a necessary first step, and this can be difficult to do efficiently in some programming environments, or may be handled inconsistently across multiple implementations.
- A standard may have defined extension points: places in the schema where an implementer can add non-standardized content to address new requirements, perhaps in the context of an xsd:any element in an XML schema. This approach makes it possible to validate the document against the schemas in the standard, but innovation can only occur in the places where extension points are defined.
- A standard may not allow any extensions at all, and all innovation must occur through the standards maintenance process. This solves the validation problem, but is not always a realistic approach because of the slow pace of standards maintenance.
MCE is designed to address the limitations of some of these alternative approaches, and it has shown great promise for the types of issues that arise in the evolution of document formats. Each implementer can create their own MCE-based extensions, and documents containing those extensions can be gracefully handled by products that support varying sets of extension functionality (or none at all). MCE is not an unproven theory, but rather an approach informed by the things Office engineers learned from many years of dealing with these same sorts of issues in the binary file formats. We used MCE for all of the new innovations in Office 2010, and this has enabled interoperability between Office 2010 and other applications that don’t have that new functionality, as covered in the attached whitepaper. (For further information on Office’s use of MCE, see also the Microsoft Office File Formats Documentation on MSDN.)
As time goes on, of course, any new features used by more than one implementation should probably be integrated into the standard. The question of how and when to standardize these is one that WG4 have been discussing a lot this year. In October, the Japanese National Body submitted a proposal for a new multi-part standard containing standardized MCE-based extensions to IS 29500. Allowing standardized extensions inside MCE will permit implementers of the core IS 29500 standard to remain conformant while still allowing new features to be standardized.