Interoperability Challenges

I've started testing interoperability between various document-format implementations, and have found some interesting results. There are some very comprehensive tests of document format interoperability beginning to emerge (see the University of Illinois study mentioned below, for example), but the things I've been doing are pretty informal: create a document, type something into it, do a little formatting, and save it. Even though these documents are very simple, I've run into some interesting results that illustrate the challenges faced by all implementers.

First let's take a look at hidden text. Most word processing software allows users to mark text as hidden, and the text remains in the document but it's not displayed or printed. Both the ODF and Open XML formats include support for hidden text.

So I tried creating some hidden text in an ODT document. In Sun's OpenOffice (version 2.4.1), I entered a paragraph of plain text, and then I entered another paragraph of text containing the sentence "This paragraph is styled as hidden text." I then selected that paragraph, created a new style named HiddenText from the selection, and checked the Hidden checkbox in that style. The text disappeared from the screen, but when I saved my document and opened it in IBM's Symphony (version 1.2), here's what I saw:

Text that is hidden in one application and not hidden in another could be a problem in some scenarios, but in that example the content of the document doesn't actually change. Let's take a look at a more serious problem, where a chart created in one application shows up with what appears to be different data when opened in another application.

I went into OpenOffice and typed two pairs of columns of X and Y values, and then I created a scatter plot from this data. I rearranged the data series (in OpenOffice's chart UI), and this is how my chart looked in OpenOffice:

I then saved the chart as an ODF spreadsheet (.ods), and opened it up in Symphony. Here's what I saw:

In this case, the underlying issue is that the ODF standard doesn't specify exactly how multiple data series in a multi-line scatter plot should be interpreted, so different implementations will have different assumptions in this area. This is the type of problem that can be resolved in maintenance, by tightening up the specification to assure that all implementations will handle these details in exactly the same way.

These are just a couple of examples of the sorts of interoperability challenges that all implementers face when working with large, complex document format standards. We face the same types of challenges here at Microsoft, in our implementations of both the ODF and Open XML document formats. As a simple example, consider how PowerPoint handles the grid used for aligning objects: the standard states that the grid should be specified in EMUs (English/Metric Units), but in PowerPoint the units are instead to be considered 1/1024th of an EMU. If you didn't know that the unit for PowerPoint’s grid was 1/1024th of an EMU, it would be hard to write code that interoperates correctly with PowerPoint; this is why we'll soon be releasing implementation notes for ECMA-376 (as we have already done for ODF).

On a related note, consider what happens if there's a typo in the spec -- implementers who conform to that erroneous information would have problems achieving interoperability with an implementation that doesn't include the typo. For this reason, we've carefully reviewed the details of our Open XML implementation against the text of the standard, and we'll be submitting errors we've found to the WG4 maintenance process.

We're also trying to help improve ODF interoperability, in our work with the OASIS ODF TC. We have submitted 15 proposals for future changes to the ODF spec to address specific interoperability challenges we've found, ranging from additional numbering formats to table grid size to z-order semantics and many others. Working together with other implementers on IS29500 and ODF maintenance is the only way to resolve some of these issues.

Rajiv Shah and Jay Kesan at the University of Illinois have done a study on interoperability issues for open standards that focuses on various implementations of ODF and Open XML. After noting that "there are significant issues with interoperability among various implementations," they summarize their findings like this:

"We consider several implications of these results including the lack of perfect compatibility between implementations, the lack of good implementations outside of Windows, and the surprisingly good overall performance of OOXML implementations. The interoperability issues are troubling and suggest the need for improved interoperability testing for document formats. The results also highlight the importance of interoperability for open standards in general. Without interoperability, governments will be locked-in to the dominant implementations for either standard and in the process lose many of the benefits that might accrue from adopting an open standard in the first instance."

That strikes me as a very pragmatic way of looking at the issues.

In summary, to deliver on the promise of standards-based interoperability, implementers need to work together in at least these three critical ways:

• shared stewardship of the standards, through active participation in the defined maintenance processes;
• transparency of implementation, through published implementation notes that describe how the major implementations are handling the myriad details found in modern document format standards; and
• collaboration between implementers in events like the the DII workshops, interoperability working groups such as the OIC TC, and direct engagement between implementers to test interoperability.

We're working hard in each of these areas, and applying these three principles to Open XML as well as ODF. What else can we do? What do you think is needed to improve document format interoperability?

Comments

• Anonymous
  January 15, 2009
  Interesting post, Doug. Finding implementation differences between ODF apps is a bit like shooting fish in a barrel, but it’s good to see some more concrete examples out in the open. Different OOXML implementation would face the same problem (perhaps to a slightly lesser degree) – except that in the “real world” developers will have to do what MS Office does: it will become, in a purely practical sense, a “reference implementation”. For ODF this is, at least for now, not the case, as a number of implementations are jockeying for position. An interesting situation for everyone! One problem is that for some reason ODF has been taken by some commentators to be a sufficient technology for “interoperability”. It is not. So, for your hidden text example, the ODF spec says (I think this is the right bit): “The hidden text field [..] displays fixed text, except when the condition is true when it does not display anything.” (By this it means I think than an application should display - or not - the element content.) However, this is just informative text in the ODF spec. To conform to ODF an application only needs to read or write XML that is schema-valid. Applications are also explicitly exempted from supporting features: “there are no rules regarding the elements and attributes that actually have to be supported by conforming applications” decrees ODF. So both OO.o and Symphony are conformant to the ODF spec in this regard. There are no rules about how hidden text should (or should not appear). You can show it, you can hide it, you can have it in flashing pink in a new window – none of these things affect whether the application is conformant or not. It does, of course, affect how interoperable it is from the user’s POV. This is why we need better specification of the semantics of document formats. For commercial XML models we develop for publishers a lot of attention is usually lavished on the “rendering expectations” of the declared elements. A activity like that could be a good basis for going forward here – though quite how this could be couched in formal standards language, and what the best forum for doing this is, I am not sure. A lot of hard work lies ahead.

• Anonymous
  January 15, 2009
  Thanks for your comments, Alex.  Regarding the hidden text issue, I think we’re talking about different parts of the ODF spec.  I didn’t insert a hidden paragraph field, but rather created a new style and then clicked on the Hidden checkbox in the styles dialog.  So the document I created in OpenOffice 2.4 has text:display=”true” applied to the hidden text, rather than using the text:hidden-paragraph attribute.  Here I think there’s not any ambiguity regarding the semantics, because section 15.4.47 of ODF 1.1 says that if the value is “true” then “the text will be displayed normally” and if it’s “none” then “the text will be hidden.”

• Anonymous
  January 16, 2009
  Doug, Interesting results, but not really shocking given the nature of collaborative standards.   I generally agree with your suggested 'fixes', but it may not be sufficient either.  It occurs to me that part of what you and the Illinois researcher were seeing is a challenge that has ALWAYS been inherent in collaboratively developed standards: the lack of a reference implementation. The benefits to collaborative standards are pretty obvious: collaboration leads to a better spec and a more level playing field for implementers. The drawback has always been that simultaneous implementation by competitors leads to deviations (some warranted, some not) from the collaborative standard. For years, implementations of C compilers and SQL database engines have veered quite a bit from the ANSI standards to allow for the implementation of new and experimental features. Nowhere was the MORE true than during the browser wars where the much vaunted HTML "standard" bore little or no resemblance to the web pages people were creating for Netscape and IE.   Now contrast that with the evolution of the PDF specification. Adobe always made the spec available but did it simultaneously with the release of its latest version of Acrobat. The argument could be made this gave them a time to market advantage in implementing the new spec but it also provided a reference implementation so that any new entrant could be certain to build documents compatible with the market leader because the reference implementation could be used to resolve any ambiguities in the spec. The simple fact is there are no easy solutions.  Mandating the use of an open standard does not guarantee interoperability. I'm going to comment further on our own blog (follow the link above); but keep this info coming - it's nice to see some of the technical realities separated out from the political rhetoric.

• Anonymous
  January 16, 2009
  Today we've published another set of document-format implementation notes, this time for the ECMA-376