Fed up with the Tower of eBabel—all the horrors of clashing e-book formats—I joined Jon Noring in founding OpenReader. I even came up with the OpenReader name, and I’ve spent many hundreds of hours talking up OpenReader and its first implementation, OSoft‘s dotReader. A standard is worthless without good programs that use it. What’s more, OSoft promised among other things to donate its dotReader technology to a nonprofit library project in which I was and am involved.
As an ordinary e-book user I badly want be able to own digital books for real and not be at the mercy of any particular company. Libraries, schools, publishers and retailers also will benefit if OpenReader takes off. Jon felt the same about a truly nonproprietary OpenReader standard, and, I hope, he still does.
Jon’s A issue
Now, however, at the very least in terms of appearances, Jon has complicated life for us OpenReader supporters. OpenReader’s main founder and leader has revealed he’ll do business and technical development for a California tech entrepreneur named David Cote, who in fact played a role in CD-ROM standards development and was invited to participate in an important XML-related group. David’s past involvement in standards, however, does not in itself guarantee the integrity of OpenReader.
Here’s what OpenReader will need to succeed as a truly independent and nonproprietary standard despite Jon’s involvement with David C, a co-owner of DigitalPulp and related companies:
One: dotReader’s timely ability to read OpenReader files
OSoft’s dotReader needs to render OpenReader files, among other formats. OSoft says that is still in the works, but I expected to see this months ago. While OSoft is a small company with limited resources, I’m disappointed just the same. If part of the fault is Jon’s, then this is one more reason to be uneasy about his relationship with David Cote’s companies—which so far have been playing up dotReader but not OpenReader.
Whatever the reason for dotReader’s current inability to read OpenReader, please note that the OSoft promised such a capability in a statement I quoted in the TeleBlog last year.
Two: Creation tools for OpenReader
OpenReader also needs the timely and successful development of creation tools–ideally open source. OSoft was supposed to care about content creation capabilities for small publishers, in line with the same statement from September 13, 2005. I want publishers of all sizes to succeed in e-books. But small publishers can’t afford the format-conversion services that large houses use. Since Jon has positioned himself as a champion of Long Tail publishers, I’m baffled why creation tools have gotten so little attention. Jon has a million and one projects going. Perhaps he needs to cut back on the others to focus on the OpenReader creation issue.
I heartily approve of Jon’s BookX project to—yes—try to develop an open source creation tool for different e-book formats. Still, considering that BookX could take months to do, the project is no substitute for OSoft’s timely development of creation assistance for OpenReader, either directly or through arrangements with other developers. OSoft flatly me assured me that it wouldn’t neglect creation, and at the minimum here is the statement that it made for the public record in 2005: “OSoft is working on creating templates so users can easily create their own content using a pre-defined list of tags that the Reader will render. This list will be expanded with the implementation of the OpenReader standards.”
Three: Sufficient promotion of OpenReader
Why is OSoft downplaying OpenReader? OSoft promised me that it would talk up the standard along with the dotReader—and OpenReader is losing valuable time by being pushed off to the side. Consider following sentence in a November DigitalPulp press release: “OSoft’s vision is to create a documentation standard through which publishers, authors, potential authors, and readers can share, collaborate, and exchange information in one common format.” In response to questions about this, OSoft told me that dotReader was format agnostic, that it simply had an internal XML-related format into which it could translate other formats, but given Jon’s new for-profit relationship with DigitalPulp, OSoft’s partner, I’m forced to go into a “Show me” mode. The press release neglecting OpenReader is still online without a correction. Beyond making that change, it would help for both DigitalPulp’s store and OSoft to do “OpenReader coming” messages on their home pages, one way to atone. In fact, that really should be a “must.”
Now back to DigitalPulp’s press release summing up OSoft’s vision. The “common” format had better be OpenReader—unnamed in the release—or I’ll sever ties with OpenReader and no longer talk up dotReader, either. My unpaid job as strategy and external relations director for OpenReader has been to promote a nonproprietary standard, not act as a Trojan for dotReader or DigitalPulp. I’ll be happy if OSoft prospers as a result of my having talked it up as a strong implementer—I’ve donated tens of thousands of dollars of free services to OSoft, including the naming of dotReader—but my real goal, in my OpenReader job, is to promote the standard.
Also useful: A move to OASIS
Beyond the above three “musts,” I’d fervently hope that OpenReader’s standard-setting could be moved to an OASIS technical committee or similar body—in a way favoring no particular company. Supposedly that is still on Jon’s agenda. It will be fine if DigitalPulp, OSoft or both will help pay for OASIS-related activities, but I would vastly prefer that funding come other companies, too. I would also like to see an extremely transparent and fair process used to choose members of the committee without DigitalPulp or OSoft exercising undue influence.
And speaking of the possible OASIS committee, it must avoid favoritism on DRM matters, lest copy-protection be used to turn a nonproprietary format into a closed one. Finally, although I very much hope Jon will be a key player on the committee, I emphatically do not want him as its chair or vice chair, based on his less than satisfactory track record at getting OSoft and DigitalPulp to back the standard fully.
An invitation to Adobe’s Bill McCoy—a rival of OpenReader and OSoft
Meanwhile, given Jon’s connection with DigitalPulp, I’ll be inviting Adobe’s Bill McCoy, a gung-ho supporter of the International Digital Publishing Forum and a frequent critic of OpenReader, to be a regular contributor to be the main part of the TeleBlog rather than merely a commenter. I’ll encourage Bill to raise fair questions about OpenReader—which, on the surface seems odd, considering my current support of the standard, but which really is not.
Some good, vigorous tire-kicking, in public, as long as the facts justify it, can only help the cause of a truly nonproprietary standard. Jon, of course, can then respond to Bill in comments, just as Bill can respond to Jon.
Bill will also be welcome to write on other topics, including the IDPF standard-setting process, just so his posts are written in the style of the rest of the TeleBlog and either avoid arcane jargon or explain it. I’ll request both Jon and Bill to keep the number of standards-related postings reasonable since this blog is about much more than just e-book standards. I’ll not limit nonstandards postings. Bill is a “real McCoy” in his ancestor’s proud Appalachian ties, and assuming he wants to contribute to the blog (no guarantees), I’m looking forward to entertaining essays on that and other topics in an e-book context. Let the Hatfields/Norings and the McCoys both have some fun on the TeleBlog site.
Still rooting for OpenReader
Yes, I’m still rooting for OpenReader to succeed, if done right, and I’ll also hope for Jon’s business success at DigitalPulp as long as it does not imperil the nonproprietary nature of OpenReader. The standard has a lot going for it. Jon tells me he has made special efforts to allow for reliable interbook linking and provide for full-fledged interactivity, among other software-dependent capabilities, and I wildly approve of this.
There are very real differences between the OpenReader standard and the Open Publication Structure on which the IDPF is now seeking comment. OpenReader is a turbocharged version of IDPF standards, and that’s why I’m hoping that it will succeed. I hope that if nothing else, IDPF will pick up Jon’s contributions, a goal dear to Jon himself.
For Jon’s standards authoring and much more, he deserves endless praise. But that is history—what about the future?
My very most painful posting
I’ve made thousands of post to the TeleBlog, and you can bet this one is by far the most painful. I still regard Jon as a friend, and he is one of the true heroes of the e-book world—given his vigorous advocacy of e-book standards at the consumer level.
Jon and OpenReader are no small reason why the IDPF is going beyond production-level standards (even if I’m still rather concerned that vendors will use DRM to prop up the Tower of eBabel).
I hope his valuable work will continue, and I am proud so far to have been part of OpenReader.
But for Jon and OpenReader to maintain credibility—and keep me involved—it is imperative for him and others to honor the three “musts.” Ideally they’ll also move the standards-setting process to OASIS under the conditions I’ve mentioned, including avoidance of Jon as chair or vice chair.
What OSoft can do
Without giving a date, at least not to my knowledge, OSoft has said it will hold a Skypecast to discuss the future of dotReader.
It would be classy of Mark Carey, the CEO, to address the three “must” issues and the others I’m raising here. Mark and his CTO, Gary Varnell, are real talents capable of much good, and I hope that along with Jon, they’ll take my advice in the right way. If David Cote can do the same, then so much the better, regardless of the rather mixed messages that DigitalPulp has been sending out. A great way for DigitalPulp to make up for an extremely disturbing post to Jon’s eBook Community List (“Another Format? Yep, and it’s worth looking into!”) would be to allocate resources for OpenReader—to make it a genuine reality in the next month or so, perhaps in collaboration with OSoft, albeit with vendor-neutral results. As I said, what’s the point of OpenReader without strong implementations? That includes creation, not just rendering.
Normally we would have a bunch of companies implementing the standard and distinguishing themselves with extra features (which are not part of the standard). Then these extras if they prove to be meaningful would be stanrardized too, this way the standard would evolve.
In case of dotReader I don’t understand the motivation however, since it’s an open source software if they want to divert from the OpenReader format why did they come up with their own format instead of tweaking the OpenReader standard?
I think what OpenReader really needs is to educate the public. What the heck is the OpenReader specification anyway? At the end it’s rendered as XHTML, OSOft is basically embedding a web browser control and adding some nice features around it, like highlighting and commenting.
I’m sorry to say it but currently the direction Adobe is taking with reflowable PDF makes more sense to me.
Re: DRM…
DRM can be iplemented around any digital content (music, photos, ebooks etc), so I don’t see why the OpenReader specification should be concerned with it. If somebody wants to package an OpenReader document into a DRM scheme and finds customers willing to accept it and pay for it… so be it.
Hi, Tamas–thanks for your thoughts. In OpenReader, we’re talking about a core format providing for reliable interbook linking, greater accessibility for the disabled, and a bunch of other features not found in the IDPF specs. dotReader does not even tweak the format; for now, at least, it is not even using OpenReader. Instead dotReader relies on a native format.
Meanwhile, to add to my comments in the above post, I would encourage members of the open source community to work with Jon in areas such as creation tools for OpenReader—and, yes, implementations independent of dotReader. If they can help OSoft make good on its OpenReader promises, then so much the better. As noted, I consider OpenReader to be both valuable and fixable. It just needs to be done right.
Thanks,
David
Good to see Bill McCoy added to the group of writers. I think this weblog has been devoted to having open standards, not necessarily OpenReader (although that is an important part of it). A variety of perspectives will not hurt.
Standards compliance takes time.
As far as author creation tools, a few good XSL transformation scripts will probably do the trick (and maybe an online transformer). The problem is that companies are trying to come up single source solutions on servers, whereas individuals want the ability to do the creation/conversions on the desktop. Two different mindsets.
Hey, Robert. I’m glad you see what I’m up to. I’m just as pro “open” and pro “standards” as ever, but I do think other interpretations of them will help. We’ll hope that Bill sees the mutual benefits and will join the dialogue here.
Happy holidays!
David
Tamas: From where I sit, the primary benefit I see claimed for OpenReader is that it would eliminate the “tower of eBabel” by making it possible for software to reliably open and render ebooks in the format.
One thing required to make that promise a reality is to specify exactly how to interpret the bits you actually get when you acquire an OpenReader ebook. As you note in your comment, it’s technicaly possible to wrap up files in one format in any packaging desired, whether that’s a DRM-based package, as in the example you give, or some other packaging. But if you leave the packaging unspecified, there’s no guarantee that your promise can be upheld.
Here’s an analogy: suppose I want to somehow specify books that solve the “tower of Babel” problem for my kids: namely, ones that I’m sure they can read. (They’re beginning readers.) I might do things like certify a base vocabulary I’m sure they can understand. That’s what Dr. Seuss followed, for instance, when he wrote _The Cat in the Hat_: limited himself to 250 words chosen in advance by someone else, and wrote a wonderful rhyming book using them. But if I don’t further specify how the book is packaged, namely, rendered as marks on paper, I haven’t solved the problem I set out to address. My kids can’t read versions of _The Cat in the Hat_ that are printed in Braille, for instance, or translated into Chinese (or even Pig Latin), even if those are alternate renditions of the same original easily-read text.
So I’ve been urging the OpenReader folks for a while to make it clearer what an ebook that claims to be OpenReader-compliant should use (or not use) in the way of packaging. They haven’t done this yet, though I recognize that giving a full format or certification description takes some time. Unfortunately, I can’t find anything in the specs that gives *any* packaging format that OpenReader implementations should recognize, let alone specify what *kinds* of formats are allowed.
The link for the one example ebook given, _My Antonia_, has to tell us out of band that it’s a zip file, and it looks like we have to figure out on our own to unzip that file and look for a .orb file in the top-level unzipped directory to access the OpenReader Binder that software can then use to render the book as a whole. I can’t find anything in the actual specs that tips us off to those conventions. As a human reader, I can figure them out, but software shouldn’t be expected to guess about implicit, unstated conventions.
I do see a link given for “OpenReader Container Format Specification”, which I’m guessing will make conventions like this explicit, but unfortunately that is marked as still “in preparation” and only goes to a blank placeholder document.
I haven’t been following whatever arguments have been going on with OSoft, the OpenReader folks, and other interested parties in recent weeks. But I do know that if you want to someone to read files in a particular format, produce creation tools for it, and promote the format as an open standard, it helps to have an open and complete specification for what the format is.
It looks to me that OpenReader is not at that point yet– there are some important parts released, such as Binder and the basic content specs, but important questions like “what bits will (or might) readers get when they download a book as an OpenReader file? and how will they interpret them?” are still unanswered, at least in any public, non-proprietary specification. In the absence of some crucial parts of a format specification, there may be at least some justification for a vendor that’s dependent on a release schedule to stay in business to go their own way without waiting for those crucial details.
I don’t mean this to be an apologia for the parties other than the OpenReader group that are involved in the current dispute. I don’t know the details of what happened and why, but I hope that all of the parties can work out an arrangement that’s mutually satisfying and that upholds the principles espoused by the OpenReader group. And it seems to me that at the very least there’s some more work that can be done by the OR people that might help.
John Mark, I appreciate your comments. I fully agree it is a good idea for OpenReader to specify a ‘wrapping’ format.
And indeed we have put a draft container spec together, but it hasn’t been published yet. (It’s on my “to do soon” list.)
The OpenReader container will be fully compatible with the recently released IDPF Container. For reasons I won’t explain here, it won’t be fully conformant, but that’s not a major issue since even Adobe is using the OCF Container in “compatible” mode for containing its Mars format (PDF recast in XML.)
It is the intent, as I presently understand it, for IDPF and the OpenDocument folk (at OASIS) to settle upon a common and more generic container format. It, in all likelihood, will be almost identical to the IDPF Container because of the significant effort the IDPF Container WG put into its container spec and in developing it for containing multiple publication types. The current IDPF Container requirement that it must contain an OEBPS Publication is completely arbitrary from a technical point-of-view.
I’ll privately email you the draft OR Container spec. That document will explain how it differs from the IDPF Container spec, but for all intents and purposes is completely compatible.
Blog wars, flame wars, journalism, objectivity… personal laundry aired in public. Recently, our company, DigitalPulp Publishing was mentioned in the context of a personal dispute between Teleread Author, David Rothman and OpenReader co-founder, Jon Noring. David R. was bringing up his doubts about the ability of Mr. Noring to remain independent in the light of his new affiliation with DigitalPulp Publishing.
Ironic, that DigitalPulp Publishing was founded with the most independent of spirits… DPPstore, our eBook retailing site is exclusively for eBooks by independent authors and publishers. We are format agnostic, and hugely in favor of the development of a non-proprietary standard that will allow readers to buy eBooks with confidence knowing that their eBook will be readable regardless of their choice of reading platforms (software and hardware). David R. makes us sound like the evil empire… (I’m putting on my Darth-Vader gear now). I found myself reading this article, which expressed Rothman’s doubts of the personal integrity of Jon Noring, his ability to remain independent in light of working for and with DigitalPulp Publishing, thinking about the difference between speculative journalism, investigative journalism, and Op Ed pieces.
So many blogs are based on Op Ed… just one person blasting out their perceptions of the universe, in microscopic detail or vast sweeping statements. David R. has established Teleread with a backbone of journalistic tradition. He has focused on hardware, software, standards and other eBook issues. He has invited contributors with expertise in all of the realms of eBooks to participate in the dialogue whether or not he was in agreement with their point of view. That is why I, like many interested in eBooks, read Teleread.
I was disappointed with the tone and tenor of Rothman’s latest piece… (But admittedly, I was flattered by the notion that DPP was so powerful that anyone who joined our ranks would be swayed from all of their previously held values and beliefs). Seriously though, there is no need to air personal feelings as if they had any basis in objective facts. Also, David R. managed to muddle up the following issues:
On the first point, I agree completely with David. Corporate interests could be in conflict with standards development, IF:
DPP meets neither of these conditions. We are format agnostic in our store – we will sell eBooks in whatever format the publishers create and whatever format the consumers will buy. We are also not invested in any particular standard. We do believe that it is essential for the industry to have standards. David’s point in his article about how standards affect the consumer, “As an ordinary e-book user I badly want be able to own digital books for real and not be at the mercy of any particular company,” is a sentiment we share at DPP. We have not built a business that is dependent on the success of a particular format or an untested standard. Our business is to deliver content in the form that the consumers demand.
On the issue of Jon Noring’s honor – David Rothman has leaped overboard on this one…. Not only is Jon entirely capable of separating his vocation from his passionate avocation of standards development, it is shocking that someone so closely affiliated with him would publicly raise these kinds of doubts. In addition, as I stated previously, DigitalPulp Publishing is in favor of the emergence of a nonproprietary standard for eBook publications – leaving no moral conflict for Jon Noring to battle. In fact, standards are an essential component in realizing the full potential of the eBook market. Consumers must be able to buy an eBook reader, and know that when they download eBooks, they will be readable on their device. It’s common sense. I have seen many of the people in the eBook business engage in the folly of trying to bully consumers into a proprietary format through hardware ties or marketing schemes (such as getting a well known sales channel to exclusively sell eBooks utilizing your pet format). I believe that since there are an infinite number of solutions to the eBook puzzle, making a monopoly impossible to attain, focusing on proprietary formats only handicaps the growth of the industry. A unifying standard (which must be a non-proprietary one) will open up exponential expansion of eBook technology.
On the issue of OSoft’s timely implementation – David Rothman obviously has no idea what the process of software development entails. Timelines in software development are usually made of silly putty! The fact that OSoft has a working Beta at this date is impressive and a testament to their dedication and work ethic. OSoft’s dotReader is an open source project with SVN repositories viewable on OSoft’s Website through the Source Code Link. The development of dotReader is going to include facilitating the reading of all possible formats, including OpenReader’s own. Currently, the basic XML format (dotReader format) is just the quick and easy way to showcase the unique features of XML based formats (such as OpenReader) within the dotReader. They are NOT setting up a standard to compete with OpenReader; they have simply included a much less sophisticated XML format to get eBooks to XML quickly. We are all awaiting those creation tools that OpenReader is working on make it easier for all of us to create valid, nuanced XML-based eBooks.
The long and short – Jon Noring is a dedicated member of the standards community (a thankless job if you ask me) who is now also employed in his field of choice. OSoft’s dotReader is a well conceived and brilliantly implemented way to read eBooks (regardless of format). And DigitalPulp Publishing is a group of dedicated entrepreneurs who are grateful to count Jon Noring among our ranks, and who also fully support Mr. Noring’s continuing efforts in the standards community.
I have posted a TeleRead blog article in reply to this article.
Robert Nagle says, “Good to see Bill McCoy added to the group of writers.”
Robert, David says he’s inviting him to post to this blog — I haven’t seen anything about acceptance. And given David’s implicit restrictions (he writes, “I’ll encourage Bill to raise fair questions about OpenReader…” [emphasis added]), I wouldn’t hold my breath.
would you be interested in an ebook news site where anyone can post articles?
Hi David,
For our timely ability to read OpenReader, I suppose I am primarily to blame. I’ve been rather bogged down in bundling all of the code in such a way that it just works on all three operating systems, chasing various bugs, and hammering out APIs for our plugin systems while trying to keep this 10k lines of Perl reasonably approachable, well-documented, and maintaining about 75% aggregate test coverage. I also build the releases, manage the mailing lists, and pretty much everything else. But, this is open-source software with a very open plugin system, so I’ll pass the blame to “everyone that hasn’t sent me a plugin yet.”
Yes, software production is unpredictable. Note that we’re still planning to support many different formats and putting a lot of effort into the plugin system plus the extra code required to keep us from being married to any particular format.
At present, the only format supported is the (invented by the original java developers) xml of the legacy ThoutReader. This is because we have an obligation to existing users. However, note that this format is a *plugin* and is designed to be cleanly separated from the rest of the system. Also note that we’ll be first to admit that this format has some problems. We didn’t create a new dotReader format, and I’m still hoping that OpenReader can fill my wishlist for things like a reduced memory footprint, consistent user model, reliable cross-book linking, etc.
It might also be worth noting that dotReader is both a GUI program and a standalone Perl API. This means that any book plugin could be used as part of a conversion tool. So, converting existing thout books to OpenReader format has already been made easier by our orthogonal plugin system and modular code. If someone were to dig into this and write the code to make it happen, this would go a long way toward promoting OpenReader by making the singularly-important Content available in that format. After all, books are all about Content. The same goes for any other format-reading plugins we acquire in the future — an “anything to OpenReader” converter in 600 lines of Perl is a very reasonable goal.
I haven’t seen many examples of openreader format. Is there a test suite? With any sort of non-trivial extensible format, one can never prove 100% support of the specification. The only thing to do is test, test, test. Take a look at the thout format samples in http://svn.dotreader.com/svn/dotreader/trunk/test_packages/ for an example of the sort of thing required. Most of these are very small books that flex some specific aspect of the format — then there is a corresponding test program in the t/book directory which loads the book and verifies the plugin’s behavior.
To make it easier for us and others to support OpenReader, it would be extremely useful to have a similar collection of miniature books (most of mine read like “ABCDE”) which express the particulars and edge cases of the format. These will also help in nailing down the specification and serve as examples for authors and developers of content creation/conversion programs. If you send me a username and password on a secure channel (or the output of `htpasswd -nsb yourusername yourpassword` in e-mail), I’ll give you a section of our subversion repository to post the samples in.
And yes, we do have limited time. I assure you that OpenReader is still on the todo list, but I have a very long todo list.
I’ll also go on record here and elsewhere saying that if you send me working code and tests, I’ll ship it. I can’t stress enough how open this open-source project is. Join the developer’s mailing list (http://dotreader.com/mailman/listinfo), send patches, tests, documentation, plugins, examples. As long as clocks only turn clockwise, we’re going to need community to make dotReader the open standards vehicle that it is intended to be.
Thanks,
Eric Wilhelm
dotReader architect / chief developer
Hi, Eric. Many thanks for the detailed note. As noted earlier, I can appreciate the vagaries of development. I’ll let Jon respond to your needs-related questions. I certainly would encourage members of the e-book community to help both you and Jon in OpenReader-friendly ways. I heartily approve of Jon’s BookX efforts.
Meanwhile perhaps you can remind OSoft of the need to publicize the forthcoming OpenReader capabilities—that will make it easier to attract volunteers.
An “OpenReader coming” logo on the home pages of the OSoft and dotReader sites would be most appreciated.
This would help satisfy the agreement that OSoft made originally with Jon (on behalf of OpenReader) and me (on behalf of LIbraryCity).
I realize you’re dealing with technology, not promo, but the two areas interact. If Mark publicizes and talks up OpenReader and positions the current approach as a temporary fix, then it’ll be easier to attract volunteers and clear up the confusion that Tamas and others understandably have.
Thanks,
David