Cutting corners - the realpolitik of ODF standardisation?
I made a comment at an event in Brussels and quite a few people have asked for more information so I thought I'd post here.
I mentioned that in my opinion, Sun were completely aware that ODF wasn't sufficiently defined to support spreadsheet interoperability as long ago as February 2005 and that the realpolitik inside OASIS was to take advantage of the EU IDA's request to standardise by rushing to be first despite knowing the ODF specification was deficient in at least this area.
So, here's how I reached this opinion. As I said, I'm delighted to be corrected if this proves to be factually inaccurate. This isn't intended to be FUD, I am only quoting OASIS' own mailing list.
Back in early February 2005 James Clark made a comment to the OASIS OpenDocument technical Committee about the lack of interoperability for spreadsheet documents (my emphasis)
I really hope I'm missing something, because, frankly, I'm speechless. You cannot be serious. You have virtually zero interoperability for spreadsheet documents. OpenDocument has the potential to be extraodinarily valuable and important standard. I urge you not to throw away a huge part of that potential by leaving such a gaping hole in your specification.
Claus Agerskov further commented that this provided a means of creating lock-in (my emphasis)
"OpenDocument doesn't specify the formulars used in spreadsheets so every spreadsheet vendor can implement formulars in their own way without being an open standard. This way a vendor can create lock-in to their spreadsheets"
SUN's Chair of the Technical Committee, Michael Bauer, responded that
"There are from our point of view also no interoperability issues, because the namespace prefix mechanism we have specified unambiguously specifies what syntax and semantics are used for a formula".
which David Wheeler interpreted as saying
"Every implementation must reverse engineer all other implementations' namespaces (they're not in the spec, so everyone's free to invent their own private incompatible namespaces). Then, every implementation must implement all the syntax and semantics of all other implementations' namespaces for formulas, if they wish to achive interoperability. And oh, by the way, your implementation might not implement the namespace for the document you're trying to load, so you may lose all the formulas."
David made some proposals which I believe have now led to the OASIS Open Document -Formula subcommittee, but at the time the OASIS Technical Committee answered David saying amongst other things that: (my emphasis)
"... please keep in mind that we have to fit proposed solutions into the politic of work that has already been done. A politic that represents years of work that is just now on it's way to ratification at OASIS, and beyond to ISO. Keep in mind also that the ISO certification comes at the request of the European Union. Time is of the essence. Ratification perhaps trumps perfection. At least for the moment. We are very much aware that whatever we leave outside the specification remains open (or not) and exposed to ambiguities and custom implementations, all of which have proved to be so problematic in the past."
Remember this was back in February 2005. In September 2005 Newsforge carried an artlicle euphamistically titled "OpenDocument office suites lack formula compatibility" but which did a good job of demonstrating that OpenDocument represented a step backwards from the status-quo and included some screen-shots to illustrate the problem, concluding
Sooner or later, a solution to the formula incompatibility problem will be found. Ideally, someone will solve it sooner, and without any intellectual property problems. The capability to exchange spreadsheets freely is essential for free desktops in the business and education markets, and it should not be limited by artificial restrictions.
Now when Tim Bray (also with Sun) heard of this in October 2005 he wrote
Bad Formula Trouble · I learned, to my dismay, that the ODF specification is silent on spreadsheet formulas, they’re just strings. This is obviously a problem; much discussion on what to do ensued. I lean to the idea, much bally-hooed by Novell, of simply figuring out what Excel does, writing that down, and building it into ODF v.Next. Mind you, anyone who’s really been to the mat with Excel, in terms of Math & Macros, knows that it isn’t a pretty picture, there are real coherency problems. But it’s good enough and the world has learned how to make it work.
I don't know if its correct, but the roadmap on OASIS' web site seems to suggest that this might be fixed in ODF 1.2, but it seems that's not due before October 2007 at the earliest.
Remember all of this was known back in very early 2005, pre-ISO certification. Since ODF became ISO26300 OASIS has been working hard to promote its adoption. A prime tactic has been to create the impression with public policy makers that the longevity of public records is in danger unless they mandate ODF.
Somehow they neglect to mention that the spreadsheets can't interoperate though.