EchoDitto Blog

Open Source and Inventiveness

Tim Lee takes issue with an article by Nicholas Carr in which Carr explores the limits of the open source development methodology. Carr says the OSS isn't very good at innovating. Tim disagrees:

There are lots of examples of ground-breaking free software. Apache wasn’t the first web server, but it preceded most of the proprietary servers now on the market, and it’s been the market leader almost from its introduction. Perl, Python, PHP, BitTorrent, and SendMail are a few examples off the top of my head of free software programs that aren’t clones of proprietary protocols. And on the flip side, a lot of Microsoft software is derivative. For example, Excel, Word, Windows, IIS, and SQL Server were all late-comers to markets that had been pioneered by other companies.

The reality is that really groundbreaking software ideas — word processors, spreadsheets, the web — are extremely rare, and they’re usually the work of a single smart individual. Whether that individual chooses to commercialize his idea or license it as free software is up to the whim of that individual. The vast majority of software development, on the other hand, involves making incremental improvements to existing software products and concepts. As far as I can see, that’s equally true whether you’re talking about free software or proprietary software.

I think Tim's right: it's silly to complain about the lack of paradigm-shifting projects on SourceForge. Ideas like the nonlinear video editor, the word processor or the spreadsheet simply don't come along all that often, and many of them originated before the internet had made open source software possible and widely-known. And, in fact, I think you'd be surprised how many experimental interfaces you could find in the open source world if you cared to look. It's just that most of them are pretty lousy. Boring as it may be, I believe that there is a relatively narrow range of visual metaphors that are appropriate for manipulating data on a 2D display with a pointing device and a bunch of buttons.

With that said, I do think Tim's overstating his case just a bit. There aren't enough originality-related datapoints within the world of classical applications — instead, let's look at games. The evidence here is less than encouraging. There are plenty of open-source games, but few have gained traction in a serious way. And they're frequently derivative: the most successful open-source games tend to be clones of existing commercial games like Tetris or Civilization, or applications that build on or otherwise support playing commercial games more cheaply or easily.

But I don't think this lack of originality is due to any inherent flaw in open-source contributors or the organizational model they employ. I think it's simply a question of capital — open source projects typically haven't got any. The vast majority of applications benefit from network effects that arise when their userbase becomes large enough: suddenly it's easier to find someone to play against online, or the documentation is better, or you can exchange files in the same format that your friend uses. It's relatively easy for open-source projects to achieve the necessary level of market interest when dealing with highly technical users and applications, as Tim's examples demonstrate — there are accepted techniques (e.g. the RFC process, making frequent commits to the project) and media outlets (e.g. listservs, usenet) that can confer legitimacy and generate interest without an investment.

That isn't true for products aimed at consumers, however — these frequently require investment in concerted promotion efforts in order to reach the network tipping point (or else must meet an unfulfilled need in a way that generates earned media like Wikipedia has). Witness the Ogg Vorbis audio format, which is widely agreed to be technically superior to MP3, offers advantages that are applicable to the average consumer, and has been almost completely ignored.

This brings up another issue: collaboration with the private sector is often much more difficult for open source projects. If Ogg had been able to attract support among portable audio player manufacturers its history might be very different. Similarly, it's easy to imagine that if the Blender project could talk to Nvidia about what their next generation graphics cards would offer, it might be able to better-compete with professional products like Maya and 3DS Max (I should note that I'm not very familiar with 3D modeling software, so perhaps I'm wrong about this).

But that's frequently not possible, and consequently the open-source projects that succeed are the ones that can form a symbiotic or parasitic relationship with other products with institutional backing. It's a lot easier to get folks to use your word processor if it can speak to Microsoft Word; it's a lot easier to get people to play your FPS if it's an add-on to a game that already has a large installed base; it's a lot easier to launch an IM client if it talks to an existing network.

That's the issue, I think. Carr makes out open source software's derivative nature to be a question of organization, but that's flatly wrong. The conventions may be different, but many projects have well-developed protocols surrounding the use of development IRC channels or mailing lists. Anyone who thinks the Mozilla or Apache (or Drupal!) projects could have achieved what they have without a high degree of coordination among volunteers is crazy. These organizations have roadmaps, design documents and established internal policies. It's just PR money and private-sector legitimacy that they lack. Fortunately the public's increasing technical literacy means that the need for the former is declining; the undeniable commercial relevance of open source means that projects are increasingly being accorded the latter.