Open XML Implementations Part 2

There is a great conversation happening in the comments section of my last post about independent implementations of Open XML. Andy Updegrove asked a few questions (and later made some observations) along with a group of folks who are actively building implementations. I'd encourage you to read that in conjunction with this.

Topics to tackle today (in as few words as possible) based on the earlier thread:

1. What Is the utility of the Open XML specification in terms of implementation? Is that utility to be measured solely on the result of a competing office suite to Microsoft Office? Or are there other possible benefits?
2. Are implementations of Open XML that were expressly done to interact better with MS Office somehow "less" than implementations of Open XML in a form completely unrelated to Microsoft Office?
3. If there are indeed applications stemming from the availability of the Open XML specification that do not have anything to do with MS Office - is it important that they be backwards compatible with MS Office old binary formats?

The Utility of Open XML

It is important to remember that the European Commission (yes, I am quite aware of the CFI decision, and very aware that we have some more work ahead of us to meet expectations on interoperability) expressly recommended that Microsoft standardize the MS Office document formats. This happened in conjunction with our preexisting work on XML formats, customers requesting more openness in formats, the Massachusetts ETRM policy, ISVs looking for more ways to work with XML and our formats, etc., etc. My point here is that many, many people felt there would be great benefit to the standardization of Open XML.

The utility of a fully documented Open XML specification (commented issues and all) is manifest in how the individuals and organizations are making use of it. To me, this is a critical part of the discussion. The desire to have greater openness (I have written about this before) has led directly to increased choice, and a wider range of available solutions. The specification has created new economic opportunity which will ultimately lead to customer benefit. 

These implementations are directly competing with Office while others are utilizing it to work more effectively with MS Office, and there are those that have absolutely nothing to do with MS Office. This is exactly what was being looked for in requesting the format be standardized in the first place.

Working With MS Office - Good or Bad?

There is no reason to disparage an implementation of Open XML for working with MS Office or not. The value of those implementations will play out in the quality of the overall solution in which they are part. This is a big part of what standardization is all about. Does the specification result in market-viable implementations. OSI vs. TCP/IP ring any bells? Standards bodies are constantly on the lookout for work items that are relevant to the marketplace, and offer high value to implementers.

If working better with MS Office were the only way that implementations of Open XML worked, it would be hard to argue that there was no utility in that, nor that it did not create massive economic opportunity for the implementers. Better yet, this is not the case. The opportunity is considerable for non-Office-related work as well as for those with a direct relationship.

Backwards Compatibility

Backwards compatibility with old MS Office binary formats remains a core tenet of the Open XML work at Ecma. Vast amounts of data is stored in the old binary formats and TC45 is clear in the importance of maintaining the bridge back to that data.

But, is it absolutely critical for any app that uses Open XML as its data format to have high-fidelity backwards compatibility? Especially if it has zero relationship to the MS Office suite, or even to any recognizable office automation function? Probably not, but who am I to judge? It may be that they want to take streams of old Excel data into some new-fangled solution - ok. In my book, the option needs to be there, and certainly in the case of the Microsoft Office 2007 implementation it is something that our customers have told us is critical. Outside of MS, I can imagine a document management company building with backwards compatibility in mind, while someone doing a vertical market data analysis solution with a stand-alone reporting engine that pumps out Open XML dynamically to not care a whole bunch about it.

Implementation is often used as a measure of the viability of a standard. For those National Body representatives looking for clear evidence that Open XML is already a successful standard, is already providing the exact opportunity and choice it was supposed to - just observe the comments made to my last posting. The specification most certainly will be improved by the work done considering all of the comments. Maintenance will continue to improve it as well - and that will be done (hopefully) under the proposed joint maintenance agreement with SC34.

Just today I heard of 2-more completely independent implementations in France, plus an additional 8 or so doing improved integration with MS Office.

The fact that there are so many credible, and high quality implementations merits some deep consideration in the ballot resolution process this coming February.

Comments

• Anonymous
  September 18, 2007
  The comment has been removed

• Anonymous
  September 18, 2007
  Swashbuckler - The recommendation was for us to standardize our format...and we did. For this standard, it was important that we work to bridge the gap between traditional standards work and free software in terms of IP...we did. It was important that the specification be complete enough for independent implementations...it is. It was important for the work to be done by a group that represented broad interests in the industry (partners, competitors, public sector, customers)...it was. I respect your passion, and point of view. I just don't happen to agree with it. That is not spin, that is simply someone with a different opinion than you. Microsoft Office has enjoyed significant success in the marketplace. With that success comes benefits and challenges, like anything else. Backwards compatibility as the innovation continues to push the product forward is a serious challenge. As for your question about my thoughts on people implementing the whole standard, I have a simple answer. It is their choice. The terms that govern the IP are structured on purpose to enable partial or whole implementation. I would assume you don't want us making that decision for you. I also remind you that you don't have to implement it at all - that is a choice for you as well. The individuals who have been commenting on the other thread have chose to implement Open XML for a variety of reasons - and some even clearly state that they made this choice over ODF for technical or business reasons. Others will implement both. GREAT! I love it. That is the funny thing about supporting choice...is endorsing that actual choices that people make. Finally, I have never endorsed the idea that the things in the Open XML specification that could use improvement should be ignored. In fact, I have been publicly endorsing the idea that ALL comments SHOULD be considered, and the Projet Editor/TC45 team should do the heavy lifting to make the spec better based on the comments. That is common industry practice (ODF 1.1...ODF 1.2? Anyone? Bueler...Bueler? Anyway - we are not likely to agree, but please keep me honest. Thx Jason

• Anonymous
  September 18, 2007
  "The recommendation was for us to standardize our format...and we did." Can you cite the exact text in the document you linked to that says this?  What I see says "Preferably, these open document exchange and storage formats would be subject to formal standardisation via international standardisation procedures."  That's NOT the same as recommending Microsoft standardize OOXML. "As for your question about my thoughts on people implementing the whole standard, I have a simple answer. It is their choice. The terms that govern the IP are structured on purpose to enable partial or whole implementation. I would assume you don't want us making that decision for you." Ah, but you are making that choice for me because all of the issues regarding backwards compatibility are not documented in the standard. Fortunately, I doubt either I or my employer will have to worry about it, but I'm sure others do. "I have never endorsed the idea that the things in the Open XML specification that could use improvement should be ignored." Kudos!  Very carefully crafted response! Put another way, you are in effect recommending that serious bugs be deferred (serious being in the eye of the beholder) to a subsequent version of the spec. Companies always have to make that kind of determination when it comes to ship software.  "Is this bug serious enough to stop shipping?"  Standards should be held to (if you'll forgive the pun) a higher standard. "That is common industry practice (ODF 1.1...ODF 1.2?" It seems you are conflating enhancements with bug fixes. Let me ask you this, if ODF vX would submitted for approval with all the problems that OOXML has, would you honestly want to see it approved?

• Anonymous
  September 18, 2007
  I think we are going to simply have to agree to disagree on this one Swashbuckler. I do hear you on your points, and I do not consider them without merit. I don't agree with the conclusions. I did a longer posting on the status of ODF when it went through the JTC1 process. There were errors in the spec, and things that were completely missing (formulas, right-to-left text), and there were editorial problems (just read the comments). I did not oppose it then, and I don't now. If I am to take the position of choice on document formats, then I should be consistent shouldn't I? The business strategy behind ODF was to standardize fast with an immature spec, get the ISO impramature to drive awareness, use the ISO status to try and drive procurement preferences, and start fixing all the things that were either shoddy in v1 or not in there at all. That strategy seems to have done quite well for its supporters up to this point - but let's call it what it is. ODF was not about the purity of the spec, nor of the purity of ideals about doc formats. (that is not to say there was not good engineering, nor good ideals - it is just that they were not the real motivators). At the BRM - it is not about a referendum on ODF. It is about Open XML. I think the fact that these implementations exist should be an influential part of the long-term view of Open XML.

• Anonymous
  September 19, 2007
  The comment has been removed

• Anonymous
  September 20, 2007
  The comment has been removed

• Anonymous
  September 20, 2007
  Also interesting: At the time of those tag recommendations called: "TAC approval on conclusions and recommendations on open document formats" OASIS still called their format Open Office formats and their TC doucments were signed with Open Office TC. So the EU recommendations are not just the basis for standardization efforts but also for the  name change of OASIS' Open Office format and TC to Opendocument.  

• Anonymous
  September 22, 2007
  Jason, I've posted an entry back at my blog pointing back at this and your last entry, and agreeing that what really matters is what the marketplace chooses to do with OOXML and ODF, with some additional observations on how that relates back to the goal of assuring long-term access to documents. Andy Updegrove

• Anonymous
  September 22, 2007
  The comment has been removed

• Anonymous
  September 23, 2007
  The comment has been removed

• Anonymous
  September 23, 2007
  Matusow said: Vast amounts of data is stored in the old binary formats and TC45 is clear in the importance of maintaining the bridge back to that data. Let's suppose I want to implement to complete OpenXML specification on non Windows platform, including the ability to work with old  Word .doc binary formats. Is there a document, describing the old binary format, available ? If it isn't, then  I would like to know  how can I implement the complete ECMA OpenXML standard (and, perhaps the complete ISO OpenXML standard in the future )? I am assuming, I am missing something here.

• Anonymous
  September 23, 2007
  While I generally have no objection to your posting here this bit struck me as a play on words (marketing speak if you will) "If working better with MS Office were the only way that implementations of Open XML worked, it would be hard to argue that there was no utility in that, nor that it did not create massive economic opportunity for the implementers. Better yet, this is not the case. The opportunity is considerable for non-Office-related work as well as for those with a direct relationship." The idea of standards is to allow for interoperability between different products or offerings, this is a very tangible benefit to consumers and developers/manufacturers alike..unless of course they are already in a dominant position. If the implementations of OOXML are primarily just allowing for file compatability with Microsoft Office I would argue that it doesn't bring true interoperability and as such fails to fulfill the role of a standard. In that case its somewhat comparable to a alternate history where the 802.11 standards were for the purpose of interoperating with for instance Linksys wireless routers. Can you give me a number or proportion that specifies how many of the implementations are for purposes aside from Office interoperability (because I know Gnumeric isn't) and how many are full implementations? The reason being that surrounding this entire standardization process there have been a number of announcements such as of "strong support" and "implementations" that were less than complete. I have no objection to a standard if it allows complete interoperability, but if theres not a single case of a full implementation aside from those using Microsoft API I do not believe that ECMA376 passes that test.

• Anonymous
  September 23, 2007
  It is well worth noticing on the above comments  that ODF with almost two years headstart in standardising still has no full implementation and also no fully interoperable implementations with OpenOffice which is the main implementation. It is very well possible that with another 1,5 - 2 years the amount of interoperability between the different Office Open XML application excedes the current interoperability of ODF implementations.

• Anonymous
  September 24, 2007
  Woods, I agree with your comments. However, I think publishing something as a standard, even if in real life it is simply to interoperate with one program and one only, has merits. This is true even if one vendor is in charge of the specification and they take madman-like dictatorial approach. I just don't think these type of standard deserves ISO blessing.

• Anonymous
  September 25, 2007
  Swashbuckler , > Ah, but you are making that choice for me because all of the > issues regarding backwards compatibility are not documented in > the standard. If you seriously think that backwards-compatibility is solely concerning the couple of handfulls of legacy attributes, you ought to do a better job at your homework. Backwards-compability is much more than that and it cannot be removed from the spec. It is the ability to persist the possible settings in the previous formats in OOXML, and stating that it only concerns the legacy attributes really show the debth of your technical knowledgde on OOXML.

• Anonymous
  September 25, 2007
  hAl, My question and general push in the prior posting is not so much has it been fully implemented (I would be surprised if it had due to the refusal of the largest player to accept it), the question is can it be? While I've not tested Lotus Symphony yet to see if it implements complete interoperability (and that would probably take some time) theres none of the questionable patent assurances and undocumented or under-documented format specifications. If Microsoft would release these specs and make an assurance that an independent implementation of them by third-parties would not be acted upon legally then in one major respect the objections I have to the standard would disappear. Because of the reliance by OOXML on a number of proprietary Microsoft technologies the problem presently is the same we've had with the Microsoft binary formats. The format is not vendor neutral and there exists no assurance that files will remain accessible indefinitely.

• Anonymous
  September 27, 2007
  [quote]If Microsoft would release these specs and make an assurance that an independent implementation of them by third-parties would not be acted upon legally then in one major respect the objections I have to the standard would disappear.[/quote] That is what Microsoft have already done.

• Anonymous
  October 02, 2007
  @Samuel Woods: Lotus Symphony is based on the OOo codebase, so it's hard to say that it's "independent."   Also, you should note that Microsoft has granted all the patents needed to implement the OOXML document set.  Furthermore, it should be mentioned that Microsoft does not regularly sue people over patents...  I've only been informed of one case of them suing a hardware manufacturer who was using unlicensed technology while its competitors had chosen to follow the law and bear the costs of licensing.  If you can find another lawsuit with Microsoft as the plaintiff, I'd love to know what it is.

• Anonymous
  October 04, 2007
  The comment has been removed

• Anonymous
  October 04, 2007
  The comment has been removed

• Anonymous
  October 10, 2007
  @Dave S. I have not requested the binary format specification but I know of people who have requested them and have recieved those by post. Dave, the binary specs are likely not up to scratch for a standards and with billion of documents out there MS would certainly not go to a standardization proces which can change/alter the format and would only lead to extra conversions or to MS not supporting the official but altered standard version. Or would you know of a standards organization that would standardize the format as is (so not standardization proces). So standardizing the binary formats is not even a remote option. We only want one conversion and that is from MS binary to an open standard XML format. Nobody would beinterested in some extra conversion layer from MS binary to open standard binary. So having the format specs available for everybody in free use in any implementations is likely the next best thing

• Anonymous
  October 16, 2007
  The comment has been removed