Tuesday, July 10, 2012

Will Open Source Software Someday Rule the World?

I've been listening to the ChangeLog and reading a lot about the explosive growth of open source software. There is no doubt that this phenomenon is creating innovative products and exploring new ways for creative individuals to self-organize. I forgive young software engineers for thinking this is a complete game changer and that all software in the future will be developed using this model. Perhaps they're right but I don't think so. I want to explore this assertion and see what the data shows. Let's start with the Apache Foundation.

In their how-it-works page, they describe how the Apache server source code was taken from the NCSA HTTPD but was orphaned when the original developers moved on to other projects. Brian Behlendorf created a mailing list of other admins using the product to communicate patches on that code base. That original group continued to organize and formalize. They then began to sponsor allied projects. All the projects were organized to grant authority to volunteers after they had proven their abilities; a culture of meritocracy. This group was not only users of the products, they also had the skills to modify the product and the reasons to do so. Their working lives revolved around the functions of these products and many changes were either self-serving or would please they people who they served.

These open source projects create a resource that is designed as non-excludable and non-rivalrous. Anyone who find the repository is welcome to download the software product and use it. The use of the product by one person does not preclude the use by anyone else. In many ways this is the classic public goods model. What is interesting is how the FLOSS projects do not seem to suffer from the free loader problem or any form of tragedy of the commons.

Someone who downloads a software product from a FLOSS (Free/Libre Open Source Software) repo will often have access to a significant asset. The Apache web server is a sufficient example. This product competes quite well against products from for-profit enterprises and could possibly attain a dominant position in the marketplace, if it hasn't already. But to achieve and maintain a market position like this required a significant investment in effort at one point. In the case of the Apache webserver it was NCSA. If the creation of a superior product that is then gifted to the public, this matter would be easily understood. The Apache webserver was not a single creation that was gifted since the needs of webservers have changed dramatically over the years. In fact it was the new needs that motivated the creation of the project in the first place. Since the product was already a reasonable base upon which to build, and since the needs affected many users of the product, a natural mutual interest was created and nurtured by the communications channels that were available. This is still the model for most people of a FLOSS project.

If the common asset were open to modification by anyone who had access to it, it would surely have been doomed by the tragedy of the commons. Each person who thought they knew what they were doing would have made changes and some of them would have broken the system. The form of ownership that evolved was that only a subset of people would have commit rights and they already had established enough trust in each other to be assured that they would act for the common good of the project. Since these "managers" depended upon the product, their interests converged.

This software, unlike physical public goods, could be consumed without limit as it existed. But to remain viable in the marketplace, it needed to evolve to meet the changing requirements, lest it become obsolete. Here is where the Apache project had a distinct advantage; each consumer also had the requisite tools and knowledge to adapt the product for the changing needs. These changes were insignificant compared to the complete body of code and the cost of creating each patch was also insignificant to the cost of changing to another product. Once the patch had been created, the marginal cost to share it with the community was insignificant and its value to the individual was limited unless that developer/user chose to take on the responsibility for creating a marketable product from the FLOSS base. Since this was not compatible with that person's core mission (we assume), they chose to contribute. Since each person was acting in their self-interest, the collective effort not only allowed the product to evolve to meet new demands but could also spawn new products as needs diverged.

So what are the elements to create a successful FLOSS project? First, there must be some base of code that provides enough out-of-the-box functionality to answer some need by a user. Second, there must be some core set of users who have a shared vision of what the product is and can agree on commits to that product. It seems as if this core group must initially be the primary source of commits as well as be willing to support new users of the product. Third, the end-users of the product must also possess the tools, skills and motivation to patch the product when it does not meet their needs and offer these patches back to the project. It is possible that the work of Ronald Coase may be helpful in the analysis of FLOSS projects.

The FLOSS market has changed over the years and it appears that other models are appearing. In particular the market for ERP software in FLOSS looks very different. In this case, the initial investment to create such a product is much higher and the market much thinner. What organizations now seem to try is to offer a "community version" of their product which is appealing to some subset of the market and then use it as a loss leader or teaser to find others who might be willing to purchase a traditional license for the full-product.

Another model I see in the repos is people who have come up with a relatively small widget which they have gifted. It is not clear if their hope is to spawn a new open source project from this seed or whether they are just engaging in a form of gifting. My instinct tells me the chances of a relatively small product spawning a viable new project is small. First, it may make more sense for the the consumer to merely copy the code into their own repository rather than take the product in as a component. If that is the case, the opportunity to achieve a back channel of patches to the product will be undermined and the product will remain static. If there are any changes in the needs, the product is likely to wither from the lack of contributions to keep it viable in the market. Second, without a core of developers who see value in providing the oversight to the project that is needed to avoid code rot, the burden will fall entirely upon the original developer. If that developer does not make frequent mods to the code herself, it is likely to become a burden since there will be a limit to the time that the developer will devote to that one product.

Eclipse seems to be following a different model where instead of individuals supporting the product it is organizations. This is an area I need to research since it is not clear to me how IBM benefits from the expense in Eclipse.