Posts Tagged open source

The economic value of Open Source software

(this is a repost of the original article with some corrections. Includes the Oxford TransferSummit 2011 presentation based on this data)

What is the real value that Open Source has brought to the economy? This is not a peregrine question. Since most of the current evaluation methods are based on assessing “sales”, that is direct monetization of OSS, we are currently missing from this view the large, mostly under-reported and underestimated aspect of open source use that is not “sold”, but for example is directly introduced through an internal work force, or in services, or embedded inside an infrastructure. Summary: OSS provide cost reduction and increases in efficiency of at least 116B€, 31% of the software and services market.
World business. 3D image. The isolated illustration

Getting this data is, however, not easy. There is an easy approach, called “substitution principle”, that basically tries to measure how much a collection of hard-to-measure assets is valued by counting the sum of the money necessary to substitute them; for example, counting the value of all the Apache web servers by adding the cost of changing them all with an average, marketed substitute. This approach does not work, for two reasons: first of all it introduces a whole world of uncertainty, given the fact that software is never perfectly exchangeable with an alternative. The second is related to the fact that users may be unwilling to pay for an alternative, so from that point of view the real value is much lower. This is, by the way, the (erroneous) way that RIAA and other rights organizations measure piracy losses: by counting how many times the copy of a film is downloaded, and assuming that all the people that downloaded it would have paid for a full cinema ticket if piracy did not exist. It is obviously wrong – and would be equally wrong if we applied the same principle.

Another approach is to measure the revenues of companies that are adopting an OSS-based business model, something that we have extensively studied in the context of the FLOSSMETRICS project. The problem with this approach is that it totally ignores the work that is performed without a monetary compensation, and it under-reports the software that is distributed widely from a single source (for example, the open source code that is embedded in phones or routers). A purely monetary measurement also ignores inherent improvements in value that can derive from an improved technology. Let’s make an example: let’s imagine that in the world, all television sets are black and white only, and only recently a new and improved television set can provide color. The new TV sets costs quite a lot more than the old B&W ones, so if we imagine that all the current TV viewers want to move to color the TV set provider can obtain a total amount of money that is the product of the cost of the new TV set multiplied by the number of viewers. The company is happy :-)

Now, let’s imagine that a magic signal allows the old TV sets to show color images. The company that produces the color TV sets is clearly unhappy, since its value dropped instantly to zero, but on the other hand all the people with B&W TV sets is happy, even if there is no monetary transaction; the user value increased substantially. We need to capture this value as well, since a substantial amount of this economic value is hidden in the user balance sheets. This means that we need to find a different, alternative way to measure OSS value: enter macroeconomics!

We can start from the overall economic value of IT in general. There is one thing that we know for sure: the total economic value of a country or a region like Europe – 12.3T€ (trillion of Euro). We also know the average IT expenditure of companies and Public Administrations, that is 4% (source: Gartner IT key metrics data, EU eBusiness-Watch) with wide variations (small companies: around 7%, going up with size up to the average for Fortune 500: 3%). This means that the average IT spending, including services, employees, hardware, software, whatever. This means that the overall IT spending in Europe is approximately 492B€, of which 24% is hardware (source: Assinform, Gartner, IDC) – which means that software and services market is valued at 374B€. (Estimates from Forrester are in the same range, so we are at least consistent with the big analyst firms)

Still with me? Good! Now, the next step is estimating the savings that are directly imputable to open source. We have two sources: an internal source (code replaced by OSS) and external (savings reported by IT personnel through use of OSS). Let’s start with savings from OSS adoption, that can be estimated (using data from Infoworld and our data from COSPA) at 15% for “light” adopters (less than 25 OSS products used) to 29% for “heavy” adopters (more than 25 OSS products), up to the 75% of specific cases (reported by Gartner for maintenance and licensing). Taking into account the share of use of OSS in general and the variation in use of OSS among different sizes, we can estimate that the savings directly introduced by OSS amount to 41B€ – those do not appear anywhere but in the adopters balance sheets, that is in a reduction of IT expenses, or a better result for the same IT expenditure (think about the TV set example outlined before).

And now, software development. It may sound strange, but only a small part of software is ever developed for the market – what is called “shrinkwrapped”. The majority of software is developed (internally or through external companies) for a specific internal need, and is never turned into an external product. In fact, when we consider the “service” part of the non-hardware IT market, we discover that nearly half of that value is actually sponsored software development, and the remaining 35% is non-software services (support, training, ancillary activities). This means that in Europe, 244B€ are software spending in a form or the other (for example, employee wages).

What can we say about this software? We know that a part of it is Open Source, because the majority of developers (69%, according to Evans Data) is using open source components within their code. We also know, thanks to Veracode, that “sampling … find that between 30 and 70% of code submitted as Internally Developed is identifiably from third-parties, most often in the form of Open Source components and Commercial shared libraries and components”. In our own database, we found out that the role of commercial shared libraries is hugely dependent on application type and vertical sector, and it falls consistently between 15% and 30% of the code not developed from scratch. Using a very conservative model, we can thus estimate that 35% of the code that is developed overall is based on Open Source, and this means that there is both a saving (software that is reused without having to redevelop it) and a cost, introduced by the need for adaptation and the “volatility cost”- that is, the risk introduced by using something developed outside. Thankfully, we already have quite a lot of information about these costs, thanks to the effort of the software engineering community; some details can be found here for those that really, really want to be put to sleep.

Applying the software engineering costs detailed in my previous article (volatility, increased cost for code re-factoring, glue code development) we can estimate that the savings introduced by OSS are, in a very conservative way, 31% of the software-related part of the IT ecosystem, that is 75B€. The real value is higher, mainly because reused OSS code tends to be of higher quality when compared with equivalent proprietary code (data and academic references available here) but I will leave this kind of evaluation for a future article. We can however say, with quite a good level of certainty, that the lower bound of savings that OSS does bring to the European economy is at least 116B€ - the majority of which does not appear in the “market” and only in a minimal part in the balance sheets of OSS companies (consider that only Red Hat is now approaching 1B$ in revenues). It is savings and increased efficiency of companies and Administrations that use OSS, something that was already discovered: “Finally, comparing the individual data on firms with turnover of less than 500,000 euros with the variable on size classes of customers (by number of employees), one can hipotesize a correlation between the use of software Open Source and the ability to attract customers of relatively larger scale. At the same turnover, in other words, companies “Open Source only” seem to have more chances to obtain work orders from companies with more than 50 employees (ie medium – large compared to our universe of reference).” (source: Venice study on Open Source) or the fact that revenue-per-employee ratio is higher in companies that adopt open source software (on average by industry, OSS-using companies have a revenue-per-employee that is 221% of the non-OSS controls). It is also important to recognize that this is only a measure of the direct impact of OSS. The reality is that software has a substantial impact on revenues; an example I found out is Siemens, that with 70B€ in revenues spends 5% in software drives 50% of its revenues. A similar impact can be expected of the savings introduced by OSS – something that we will talk about in a future post.

, , ,

2 Comments

It could have been different: Android, Google and all that

If there’s one thing that is totally clear, is that Android is ravaging the smartphone market, and all those that are feeling the heat are trying to use the most innovative and transparent approach to stop it: sue Google and its partners out in the oblivion. Software patents, design patents, copyrights, plain trolling- anything goes if it can help in stop the Droid juggernaut. At the same time, Google is under attack for its delay in publishing the Honeycomb source code, attacked for the half-backed results that can be witnessed in some tablet products, all of this in an environment where Android phone makers are obtaining extraordinary revenues, in large part thanks to those contested products (Samsung comes to mind).

Of course, hindsight is 20/20 as they say, and it’s easy to “predict” that the extraordinary success of Android would have generated such a defensive attack. It is however at least predictable, given the extreme litigation of software companies, that patents would have been used as a blunt instrument to fend off competitors. Could things have been different? I believe so, and I also believe that Google made some errors in its decision, especially in trying to create a control point (the “Google experience”) instead of favoring a more long-term vision.

Let’s start from the beginning. Android is two things at once; from one side it is a collection of many, many different open source projects, some external, some Google-created, some heavily modified and adapted specifically for the purpose. There is a separate, private tree (at the moment the 3.x branch) that is also based on open source projects, but is at the moment internal to Google and the partners that decided to join its Open Handset Alliance. Some projects are clearly behind their open counterparts, especially WebKit, while at the same time maintaining substantial external off-tree patches that are extremely difficult to integrate like in the Linux Kernel. There is an additional layer of totally proprietary apps, that are installable only for those partners (and phones) that subjugate themselves to an additional set of requirements and licensing rules, something that for example caused lots of problems for Motorola and Google in the SkyHook lawsuit. This will probably continue, and will be the real battleground, given the fact that Android and iOS are clearly emerging as the leading platforms.

Could it have been different? I think so. And by releasing some degree of control, Google could have created a much safer and open environment, while sacrificing very little. Let’s start with the point that what Google really, really wants is an unencumbered rich internet-enabled platform, that is not constrained by third parties, and that it uses Google services. The reality is that Google lives off advertising (at the moment, at last) and is trying to expand it to other services, like enterprise mail, documents, and so on; there are two roads to do that: the first is to create a platform that is strongly tied to Google services, so that it is nearly impossible to escape its control. In doing so, however, you face the tension of the OEM and vendors that may want to switch services, or that want to push internally developed offerings. In this case, they will have nothing left to do but to go with the competition, or create their own variant (something that already happened) increasing the adoption costs.

The alternative is the “A rising tide lifts all boats” – make it a purely open source project, where there is a real distributed control, like Eclipse. Turn it to the Apache foundation. Make all the interested partners a part of the foundation or consortium, make the code public (with limited exceptions, like for prerelease hardware drivers) and try to track as much as possible the projects where you take things from. Apply a strict code contribution regime to strengthen your position against IP claims, and especially don’t turn it into a product. Yes, you read it properly- the code should be a strict open source project. This way, it would be extremely difficult for an external party to sue the “origin”, given the difficulties in identifying a directly derived infringing device; Google could have then (using this more sanitized and cleaner base) provided insurance through a third party for IP infringement claims, if the code base adopter would want to use such an opportunity (some may decide to fight on their own, of course). This implies that an Android OEM can substitute the Google services and use something else (that, by the way, is possible even now) but would have easily prevented most antitrust-based attacks. The purely open code base and the increased external participation would have further shortened the time-to-market for providing a new board to the marketplace, reduced adoption costs, and facilitated an external ecosystem of providers for Android based services.

Google could have at least avoided some of the worst blows, increased its credibility with the various OS communities, and reduced the cost of adopting Android for an OEM, pushing more and more the platform. This, in exchange for some of the tight control that currently Google exercise on the platform. Unfortunately, I think it’s too late for that; and we will still have to face the sad situation that the life of a mobile platform is dictated purely by judges.

, , ,

2 Comments

Open Source as a differentiator?

What is an “open source company”? What is the real differentiation element introduced by Open Source? These and more questions were introduced by a great post by Matthew Aslett (if you don’t follow him, go and follow now. I’ll wait. Yes, do it. You will thank me later.), called “The decline of open source as an identifying differentiator“. It is an excellent analysis of how companies mostly stopped using the term “open source” in their marketing materials, and has a follow up (here) that provides a summary of the main responses by other analysts and observers.

The post raises several interesting points, and in my opinion provides a great basis for a more general discussion: what is the difference introduced by open source? Is there a difference at all?

Let’s start with an observation of the obvious: the use of open source to build software is now so widespread that it is not a differentiating element anymore. There goes the various “built on open source components” of some companies – practically all companies are using open source inside. It’s simply not a difference. So, let’s start with what is the real differential between OSS and proprietary:

The licensing. An open license may introduce a difference for the adopter. This means that if such a differential is used by the company, it must provide a value that derives from the intrinsic property of open source as a legal framework. For example, independence from supplier (at least, theoretically…) both in case of provider change, and independence in terms of adding or integrating additional components, even if the company is in disagreement.

The development model. The collaborative development model is not a certainty – it arises only when there is a clear infrastructure for participation. When it does happen, it is comparatively much faster and more efficient than the proprietary and closed model. For this to be a real differentiator, the company must engage in an open development model, and this is actually happening only in a very small number of cases.

In general, the majority of companies that we surveyed in FLOSSMETRICS have now a limited degree of differentiation when compared to their peers, and even as a “signaling” open source is now no more interesting than other IT terms that entered the mainstream (we can discuss further whether “cloud” will disappear in the background as well..) Of the companies we surveyed, I would say that those that we marked originally as “specialists” are the ones more apt to still use “open source” as a differentiating term, with “open core” ones the least (since they don’t reap the advantages of a distributed development model, neither the adopter reaps the advantages of the open source licensing). A potential difference may arise for development tools or infrastructures, where open source is a near necessity; in this case, the natural expectation will be for the platform to be open – thus not a differentiating element any more.

, ,

2 Comments

Composing the Open Cloud puzzle

“When it rains, it pours”: This was my first thought while reading the RedHat announcement of yet another cloud project, OpenShift (a new PaaS) and Cloudforms (IaaS). The excellent Stephen O’Grady of RedMonk provided a very complete Q&A covering the announcement and the implications quite well. This is just the last of many, many announcements in the same line, that makes the Open Source cloud environment a very active and crowded one.
-

What exactly was announced? RedHat leveraged its acquisition of Makara, a PaaS vendor with a quite interesting technical solution, and its own Jboss/DeltaCloud platform to create a combination of PaaS and IaaS, designed to facilitate deployment and management of Java, PHP, Python and Ruby applications. It falls in a market that recently saw VMWare’s entry in the market with CloudFoundry, previous entrants like Eucalyptus, OpenNebula, Nimbus, OpenStack and the still unreleased Yahoo IaaS.

It would seem that there is really too much to choose from-but the reality is that each piece does have a place, and the biggest problem in finding this place is that we forget to look at the big picture that is on the puzzle box. The first point is that there is no single market for cloud/IaaS/PaaS/whatever. There are several distinct areas, and for every one there is a potentially different combination:

  • the software (SaaS) producer, that for small scales may be interested in hosting its own small IaaS+PaaS, with the opportunity to move it or dynamically expand outside (on Amazon, for example) when bursts come in. People still think that every company has a full swing of consumption, from very little to galactic scale, while the majority of SaaS producers tend to have a very static set of customers, and acquisitions are slow (compared to procurement time) and easily managed. With a full cloud you pay an extra for scalability – the capability to grow dynamically leap and bounds, and that extra may not be economically justified. Also, many companies already invested in hardware and networking for their own in-house small datacenter, and until the amortment period is ended those companies (for financial and fiscal reasons) would not turn to a full external cloud. So, the next best thing is to create their own little IaaS (for single customer instances), PaaS (for shared customers) on the hardware they already have. The economics here is related to the reduction in management costs, and the opportunity to simplify code deployment.
  • The Telcos: my own sources indicate that lots of telcos are working feverishly to create their own cloud offering, now that there is a potential solution that has no license costs. They could have built it earlier with VMWare or Citrix, but the licensing alone would have placed them out of the market; with the new open source stacks, with a relatively limited investment it is possible to reuse the massive hardware cache already in house to consolidate and offer a IaaS to individual customers, usually mid-to-large companies willing to outsource their varying computational loads outside, or directly dematerialize their server farm. It is mostly a transitional market, that will however be important for at least 5 years from now.
  • The System Integrators: large scale INT and consultancies are pushing a set of integrated offerings that cover everything – technical, management, legal and procurement aspects for companies that are willing to move their IT completely off their backs. It is not a new model – Accenture, Atos Origin et al are doing it since the beginning of time – but thanks to the availability of open source components it does provide a more compelling economics.
  • The small VARs: there is a large number of very small companies, that targets the very low end of the market, and that provide a sort of staircase to the cloud for small and mostly static workloads. It is a market of small consultants, system administrators, and consultancies that cover that part of the market that is largely invisible for larger entities, and that is starting to move towards web apps, hosted email and so on – but still need to manage some legacy system that is currently in-house.
  • The service provider: it just started as a movement, but in my opinion will become quite big: the specialized and cloud-based service (an example is the freshly released Sencha.io). I believe that those specialized providers will be among the most important contributors to the individual OSS components, that are internally used to deliver a service, and will provide a backbone of updates for most of the ancillary parts like storage, DB, identity and so on.

And the obvious question is: who will win? In my view, if every actor works properly, there will be more than a dominant actor. The reason is related to the two main factors of adoption, that is packaging and acceleration. Packaging is the ease of installation, deployment, management and in general user and developer-friendliness of the whole. So, taking for granted that RedHat will open source it (as it did with all the previous acquisitions) the main advantage for OpenShift and CloudForms will be ease of installation for RedHat and clone users – that are still the majority of enterprise Linux deployments. It will be natural for a RHEL user to start converting some servers into a RedHat IaaS and PaaS, given the fact that systems will also fall under a simplified management through RHT Satellites; the main interest in this case will be for system integrators and developers that already standardized under RedHat. A potential future enhancement could be the push towards a better identity management (a sore point for most IaaS, with the exception of the Grid-inspired Nimbus): RedHat does have a good Java-based provisioning and identity server in-house, that can probably be expanded with external tools like ForgeRock’s OpenIM/OpenIDM. The first one that will solve the “cloud identity” problem will actually have a gold mine in its hands.

OpenStack is more relevant for bare-metal installs, and will probably get the majority of new, unbranded installs thanks to its massive acceleration – the incredible rate of change that is clearly visible by the amount of new features delivered in any release. The combination of OpenStack and CloudFoundry is probably of interest mainly for large companies that want to create an internal development environment (reducing developer friction) – in this sense, I believe that the market for secondary services will be probably limited, but with a substantial contribution from “independents” that improve the platform to facilitate their own internal jobs.

The end result is that we will still see several competing platforms, each evolving to occupy a niche and taking (and giving) pieces of code to the others. The obvious result will be the pressure on proprietary vendors, that in my opinion will have a very hard time to keep the same development pace of these collaboratively developed solutions.

, , , ,

No Comments

Nokia is one of the most active Android contributors, and other surprises

Updated: added other examples from WebKit, IGEL and RIM

Yes, it may be a surprise, but that’s the beauty of Open Source – you never know where your contributions will be found. In this regard, I received a gentle mention from my friend Felipe Ortega of the Libresoft group of a nice snippet of research from Luis Canas Diaz, “Brief study of the Android community“. Luis studied the contributions to the Android code base, and splitted the contributions using the email of the originator, assigning those with “google.com” or “android.com” as internal, and classifying the others. Here is a sample of the results:

(Since October 2008)
# Commits Domain
69297 google.com
22786 android.com
8815 (NULL)
1000 gmail.com
762 nokia.com
576 motorola.com
485 myriadgroup.com
470 sekiwake.mtv.corp.google.com
422 holtmann.org
335 src.gnome.org
298 openbossa.org
243 sonyericsson.com
152 intel.com

Luis added: “Having a look at the name of the domains, it is very surprising that Nokia is one of the most active contributors. This is a real paradox, the company that states that Android is its main competition helps it!. One of the effects of using libre software licenses for your work is that even your competition can use your code, currently there are Nokia commits in the following repositories:

git://android.git.kernel.org/platform/external/dbus

git://android.git.kernel.org/platform/external/bluetooth/bluez”

In fact, it was Nokia participation in Maemo (and later Meego) and its funding of the dbus and bluez extensions that were later taken up by Google for Android. Intrigued by this result, I made a little experiment: I cloned the full Android gingerbread GIT repo (2.3), separated the parts that are coming from preexisting projects like the Linux kernel and the various external dependencies (many tens of project – included, to my surprise, a full Quake source code…) leaving for example Chromium but removing WebKit. I then took apart the external projects, and counted Google contributions there in an approximate way, and folded back everything. You get a rough size of 1.1GB of source code directly developed or contributed by Google, which means that around 75% of the source code of Android comes from external projects. Not bad, in terms of savings.

Update: many people commented on the strangeness of having fierce competitors working together in ways that are somehow “friendly” towards a common goal. Some of my twitter followers also found the percentage of 75% of non-Google contributions to be high, and this update is meant to be an answer for both. First of all, there is quite a long history of competitors working together in open source communities; the following sample of Eclipse contributors provide an intial demonstration of that:

eclipse

But there are many other examples as well. WebKit, theweb rendering component used in basically all the mobile platforms (except Windows Mobile) and on the desktop within Chrome and Safari was originally developed by the KDE free software community, taken by Apple and more recently co-developed by Nokia, Samsung, RIM and Google:

Screenshot-Chromium Notes: Who develops WebKit? - Mozilla Firefox

And on WebKit page, it is possible to find the following list:

“KDE: KDE is an open source desktop environment and application development framework. The project to develop this software is an informal association. WebKit was originally created based on code from KDE’s KHTML and KJS libraries. Although the code has been extensively reworked since then, this provided the basic groundwork and seed code for the project. Many KDE contributors have also contributed to WebKit since it became an independent project, with plans that it would be used in KDE as well. This has included work on initially developing the Qt port, as well as developing the original code (KSVG2) that provides WebKit’s SVG support, and subsequent maintenance of that code.

Apple:  Apple employees have contributed the majority of work on WebKit since it became an independent project. Apple uses WebKit for Safari on Mac OS X, iPhone and Windows; on the former two it is also a system framework and used by many other applications. Apple’s contribution has included extensive work on standards compliance, Web compatibility, performance, security, robustness, testing infrastructure and development of major new features.

Collabora:  Collabora has worked on several improvements to the Qt and GTK+ ports since 2007, including NPAPI plugins support, and general API design and implementation. Collabora currently supports the development of the GTK+ port, its adoption by GNOME projects such as Empathy, and promotes its usage in several client projects.

Nokia:  Nokia’s involvement with the WebKit project started with a port to the S60 platform for mobile devices. The S60 port exists in a branch of the public WebKit repository along with various changes to better support mobile devices. To date it has not been merged to the mainline. However, a few changes did make it in, including support for CSS queries. In 2008, Nokia acquired Trolltech. Trolltech has an extensive history of WebKit contributions, most notably the Qt port.

Google:  Google employees have contributed code to WebKit as part of work on Chrome and Android, both originally secret projects. This has included work on portability, bug fixes, security improvements, and various other contributions.

Torch Mobile:  Torch Mobile uses WebKit in the Iris Browser, and has contributed significantly to WebKit along the way. This has included portability work, bug fixes, and improvements to better support mobile devices. Torch Mobile has ported WebKit to Windows CE/Mobile, other undisclosed platforms, and maintains the QtWebKit git repository. Several long-time KHTML and WebKit contributors are employed by Torch Mobile.

Nuanti:  Nuanti engineers contribute to WebCore, JavaScriptCore and in particular develop the WebKit GTK+ port. This work includes porting to new mobile and embedded platforms, addition of features and integration with mobile and desktop technologies in the GNOME stack. Nuanti believes that working within the framework of the webkit.org version control and bug tracking services is the best way of moving the project forward as a whole.

Igalia:  Igalia is a free software consultancy company employing several core developers of the GTK+ port, with contributions including bugfixing, performance, accessibility, API design and many major features. It also provides various parts of the needed infrastructure for its day to day functioning, and is involved in the spread of WebKit among its clients and in the GNOME ecosystem, for example leading the transition of the Epiphany web browser to WebKit.

Company 100:  Company 100 has contributed code to WebKit as part of work on Dorothy Browser since 2009. This work includes portability, performance, bug fixes, improvements to support mobile and embedded devices. Company 100 has ported WebKit to BREW MP and other mobile platforms.

University of Szeged:  The Department of Software Engineering at the University of Szeged, Hungary started to work on WebKit in mid 2008. The first major contribution was the ARMv5 port of the JavaScript JIT engine. Since then, several other areas of WebKit have been tackled: memory allocation, parsers, regular expressions, SVG. Currently, the Department is maintaining the official Qt build bots and the Qt early warning system.

Samsung:  Samsung has contributed code to WebKit EFL (Enlightenment Foundation Libraries) especially in the area of bug fixes, HTML5, EFL WebView, etc. Samsung is maintaining the official Efl build bots and the EFL early warning system.”

So, we see fierce competitors (Apple, Nokia, Google, Samsung) co-operating in a project that is clearly of interest for all of them. In a previous post I made a similar analysis for IGEL (popular developers of thin clients) and HP/Palm:

“The actual results are:

  • Total published source code (without modifications) for IGEL: 1.9GB in 181 packages; total amount of patch code: 51MB in 167 files (the remaining files are not modified). Average patch size: 305KB, Patch percentage on total publisheed code:  2.68%
  • Total published source code (without modifications) for Palm: 1.2GB in 106 packages; total amount of patch code: 55MB in 83 files (the remaining files are not modified). Average patch size: 664KB, Patch percentage on total published code: 4.58%

If we add the proprietary parts and the code modified we end up in the same approximate range found in the Maemo study, that is around 10% to 15% of code that is either proprietary or modified OSS directly developed by the company. IGEL reused more than 50 million lines of code, modified or developed around 1.3 million lines of code. …. Open Source allows to create a derived product (in both case of substantial complexity) reducing the cost of development to 1/20, the time to market to 1/4, the total staff necessary to more than 1/4, and in general reduce the cost of maintaining the product after delivery. I believe that it would be difficult, for anyone producing software today, to ignore this kind of results.”

This is the real end result: it would be extremely difficult for companies to compete without the added advantage of Open Source. It is simply anti-economic to try to do everything from scratch, while competing companies work together on non-differentiating elements; for this reason it should not be considered a strange fact that Nokia is an important contributor to Google Android.

,

29 Comments

On Symbian, Communities, and Motivation

(This is an updated repost of an article originally published on OSBR)

I have followed with great interest the evolution of the Symbian open source project – from its start, through its tentative evolution, and up to its closure this month. This process of closing down is accompanied by the claim that: “the current governance structure for the Symbian platform – the foundation – is no longer appropriate.”

It seems strange. Considering the great successes of Gnome, KDE, Eclipse, and many other groups, it is curious that Symbian was not able to follow along the same path. I have always been a great believer in OSS consortia, because I think that the sharing of research and development is a main strength of the open source model, and I think that consortia are among the best ways to implement R&D sharing efficiently.

However, to work well, Consortia need to provide benefits in terms of efficiency or visibility to all the actors that participate in them, not only to the original developer group. For Nokia, we know that one of the reasons to open up Symbian was to reduce the porting effort. As Eric Raymond reports, “they did a cost analysis and concluded they couldn’t afford the engineering hours needed to port Symbian to all the hardware they needed to support. (I had this straight from a Symbian executive, face-to-face, around 2002).”

But to get other people to contribute their work, you need an advantage for them as well. What can this advantage be? For Eclipse, most of the companies developing their own integrated development environment (IDE) found it economically sensible to drop their own work and contribute to Eclipse instead. It allowed them to quickly reduce their maintenance and development costs while increasing their quality as well. The Symbian foundation should have done the same thing, but apparently missed the mark, despite having a large number of partners and members. Why?

The reason is time and focus. The Eclipse foundation had, for quite some time, basically used only IBM resources to provide support and development. In a similar way, it took WebKit (which is not quite a foundation, but follows the same basic model) more than two years before it started receiving substantial contributions, as can be found here.

And WebKit is much, much smaller than Symbian and Eclipse. For Symbian, I would estimate that it would require at least three or four years before such a project could start to receive important external contributions. That is, unless it is substantially re-engineered so that the individual parts (some of which are quite interesting and advanced, despite the claims that Symbian is a dead project) can be removed and reused by other projects as well. This is usually the starting point for long-term cooperation. Some tooling was also not in place from the beginning; the need for a separate compiler chain – one that was not open source and that in many aspect was not as advanced as open source ones – was an additional stumbling block that delayed participation.

Another problem was focus. More or less, anyone understood that for a substantial period of time, Symbian would be managed and developed mainly by Nokia. And Nokia made a total mess of differentiating what part of the platform was real, what was a stopgap for future changes, what was end-of-life, and what was the future. Who would invest, in the long term, in a platform where the only entity that could gain from it was not even that much committed to it? And before flaming me for this comment, let me say that I am a proud owner of a Nokia device, I love most Nokia products, and I think that Symbian still could have been a contender, especially through a speedier transition to Qt for the user interface. But the long list of confusing announcements and delays, changes in plans, and lack of focus on how to beat the competitors like iOS and Android clearly reduced the willingness of commercial partners to invest in the venture.

Which is a pity – Symbian still powers most phones in the world and can still enter the market with some credibility. But this later announcement sounds like a death knell. Obtain the source code through a DVD or USB key? You must be kidding. Do you really think that setting up a webpage with the code and preserving a read-only Mercurial server would be a too much of a cost? The only thing that it shows is that Nokia stopped believing in an OSS Symbian.

(Update: after the change of CEO and the extraordinary change in strategy, it is clear that the reason for ditching the original EPL code was related to its inherent patent grant, that still provides a safeguard against Nokia patents embedded in the original Symbian code. There is a new release of Symbian under a different, non-OSS license; the original code is preserved in this sourceforge project, while Tyson Key preserved the incubation projects and many ancillary documentation like wiki pages at this Google code project.)

A full copy of the original EPL

,

4 Comments

The neverending quest to prove Google evilness. Why?

Ah, my favorite online nemesis (in a good sense, as we have always a respectful and fun way of having a disagreement) Florian Mueller is working full-time to demonstrate, in his own words, “a clear pattern of extensive GPL laundering by Google, which should worry any manufacturer or developer who cares about the IP integrity of Android and its effect on their proprietary extensions or applications. It should also be of significant concern to those who advocate software freedom.” Wow. Harsh words, at that, despite the fact that Linus Torvalds himself dismissed the whole thing with “It seems totally bogus. We’ve always made it very clear that the kernel system call interfaces do not in any way result in a derived work as per the GPL, and the kernel details are exported through the kernel headers to all the normal glibc interfaces too” (he also, amusingly, suggested that “If it’s some desperate cry for attention by somebody, I just wish those people would release their own sex tapes or something, rather than drag the Linux kernel into their sordid world”. Ah, I love him.)

In fact, I expressed the same point to Florian directly (both in email and in a few tweets), but it seems very clear that the man is on a crusade, given how he describes Google actions: “the very suspect copying of Linux headers and now these most recent discoveries, it’s hard not to see an attitude. There’s more to this than just nonchalance. Is it hubris? Or recklessness? A lack of managerial diligence?” or “It reduces the GPL to a farce — like a piece of fence in front of which only fools will stop, while “smart” people simply walk around it.”

Well, there is no such thing, and I am not saying this because I am a Google fanboy (heck, I even have a Nokia phone :-) ) but because this full-blown tempest is actually useless, and potentially damaging for the OSS debate.

I will start with the core of Florian arguments:

  • Google took GPL code headers;
  • they “sanitized” it with a script to remove copyrighted information,
  • what is left is not GPL anymore (in particular, is not copyrighted).

Which Florian sees as a way to “work around” the GPL. Well, it’s not, and there are sensible reasons for saying this. Let’s look at one of the incriminated files:

#ifndef __HCI_LIB_H
#define __HCI_LIB_H

#ifdef __cplusplus
#endif
#ifdef __cplusplus
#endif
static inline int hci_test_bit(int nr, void *addr)
{
	return *((uint32_t *) addr + (nr >> 5)) & (1 << (nr & 31));
}
#endif

or, for something longer:

#ifndef __RFCOMM_H
#define __RFCOMM_H

#ifdef __cplusplus
#endif
#include <sys/socket.h>
#define RFCOMM_DEFAULT_MTU 127
#define RFCOMM_PSM 3
#define RFCOMM_CONN_TIMEOUT (HZ * 30)
#define RFCOMM_DISC_TIMEOUT (HZ * 20)
#define RFCOMM_CONNINFO 0x02
#define RFCOMM_LM 0x03
#define RFCOMM_LM_MASTER 0x0001
#define RFCOMM_LM_AUTH 0x0002
#define RFCOMM_LM_ENCRYPT 0x0004
#define RFCOMM_LM_TRUSTED 0x0008
#define RFCOMM_LM_RELIABLE 0x0010
#define RFCOMM_LM_SECURE 0x0020
#define RFCOMM_MAX_DEV 256
#define RFCOMMCREATEDEV _IOW('R', 200, int)
#define RFCOMMRELEASEDEV _IOW('R', 201, int)
#define RFCOMMGETDEVLIST _IOR('R', 210, int)
#define RFCOMMGETDEVINFO _IOR('R', 211, int)
#define RFCOMM_REUSE_DLC 0
#define RFCOMM_RELEASE_ONHUP 1
#define RFCOMM_HANGUP_NOW 2
#define RFCOMM_TTY_ATTACHED 3
#ifdef __cplusplus
#endif
struct sockaddr_rc {
	sa_family_t	rc_family;
	bdaddr_t	rc_bdaddr;
	uint8_t		rc_channel;
};
#endif

What can we say of that? They contain interfaces, definitions, constants that are imposed by compatibility or efficiency reasons. For this reason, they are not copyrightable, or more properly would be excluded in the standard test for copyright infringement, in the abstraction-filtration test. In fact, it would not be possible to guarantee compatibility without such an expression.

But – Florian guesses – the authors put a copyright notice on top! That means that it must be copyrighted! In fact, he claims “The fact that such notices are added to header files shows that the authors of the programs in question consider the headers copyrightable. Also, without copyright, there’s no way to put material under a license such as the GPL.”

Actually it’s simply not true. I can take something, add in the beginning a claim of copyright, but that does not imply that I have a real copyright on that. Let’s imagine that I write a file containing one number, and put a (c) notice on top. Do I have a copyright on that number? No, because the number is not copyrightable itself. The same for the headers included before: to test for copyright infringement, you must first remove all material that is forced for standard compatibility, then Scenes a Faire (a principle in copyright law that says that certain elements of a creative work are not protected when they are mandated by or customary for an environment), then code that cannot be alternatively expressed for performance reasons. What is left is potential copyright infringement. Now, let’s apply the test to the code I have pasted. What is left? Nothing. Which is why, up to now, most of the commentators (that are working on the kernel) mentioned that this was also just a big, large, interesting but ultimately useless debate.

In fact, in the BlueZ group the same view was presented:

“#include <bluetooth/bluetooth.h> is only an interface contract. It contains only constants and two trivial macros. Therefore there is no obligation for files that include bluetooth.h to abide by the terms of the GPL license.  We will soon replace bluetooth.h with an alternate declaration of the interface contract that does not have the GPL header, so that this confusion does not arise again.” (Nick Pelly)

It is interesting that this comes, in and out, in many projects and several times; it happened in Wine (in importing vs. recoding Windows header definitions) and I am sure in countless others. The real value of this debate would be not to claim that Google nearly certainly is an horrible, profiteering parasite that steals GPL code, but to verify that the headers used do not contain copyrighted material, because that would be an extremely negative thing. Has this happened? Up to now, I am still unable to find a single example. Another, totally different thing is asking if this is impolite – taking without explicitly asking permission on a mailing list, for example. But we are not looking at headlines like “Google is impolite”, we are looking at “Google’s Android faces a serious Linux copyright issue”, or “More evidence of Google’s habit of GPL laundering in Android”.

That’s not constructive – that’s attention seeking. I would really love to see a debate about copyrightability of header files (I am not claiming that *all* header files are not copyrightable, of course) or copyrightability of assembly work (the “Yellow Book” problem). But such a debate is not happening, or it is drowned under a deluge of “Google is evil” half proofs.

Of course, that’s my opinion.

, , , , ,

1 Comment

App stores have no place in a web-apps world

I have read with great interest the latest Matt Asay’s post, “Enough with the Apple App Store apathy”, that provides a clear overview of why App Stores should be at the center of open source advocates’ rage. Matt is right (and some developers already started addressing this, like some of VLC project developers) but I believe that the current monopoly of app stores is just a temporary step in the wait for real web apps. App stores, in fact, do just a few things well; others not as well, and they take a hefty percentage of all transactions just because they can.
barbeque sauce

Let’s think about what an app store is about:

  • Discovery: one of the main advantages of a central point for searching applications is.. well… the fact that there is a single point for searching. Since developers, when submitting an app, need to perform a categorization or tagging it to make it searchable, an app store is actually quite helpful in finding something. Until there is too much of something. In fact, already in the iOS app store, and partially in the Android one, looking for something is increasingly a hit-and-miss affair, with lots and lots of similar (if not identical) applications trying desperately to emerge in the listing, or maybe to end up under the spotlight of some “best of” compilation. In fact, as Google would happily tell you, when you have too many things pure listings are not going to be useful; you need real search capabilities or some sort of manual suggestion (like social features, “I like it” or whatever). App stores are starting to get it, but they are insulated from the web – which means that they are unable to harness the vast, multifaceted amount of information created by tweeters, bloggers, journalists and pundits that watch and evaluate almost everything on the web. Discovery is now barely possible in a store with 100k apps; as things evolve, it will become even more difficult. In a world of web applications, well, this problem returns to a (very solvable) problem of finding something on the web. Ask Google, Bing, Baidu, Yandex, or more “intelligent” systems like Wolfram Alpha-they all seem to manage it quite well.
  • App updates: one very useful thing is the ability of an app store to send notifications of new apps, and help users in having all the update process done simply and in a single place. This is of course quite useful (just don’t claim that it is a novelty, or any YUM or APT user will jump straight at your neck), but again is totally irrelevant in a world of web apps – the app will just check for a new version, in case it uses persistent storage for caching JavaScript and includes, or simply go straight to the website of the application publisher. This also resolves the friction introduced by the app approval process in current App Stores: you submit it, and then pray :-) If an update is urgent (for example for a security fix) you just have to try as much as possible to speed it up – it is not up to the developer, anyway.
  • App backups: in a world of apps, app backups are a great idea. In a world of web apps, backups are simply bookmarks, with the cacheable parts re-downloadable in any moment. Since both Chrome and Firefox do have already their own way of syncing bookmarks, this is covered as well.
  • Payments: this is quite an important part – and something that current web apps provide in an immature way. The Google Chrome web store do something like this, but it works only on Chrome and works only with Google; there is a need for some more high-level payment scheme embedded within web apps.

As I commented to Matt in his article, I still believe that app stores are a useful, albeit temporary step towards a more open and transparent infrastructure, that we all know and love: the web. And we will not have to forfeit 30% of all revenues to be on it.

, ,

6 Comments

OSS is about access to the code

I have a kind of a fetish – the idea that source code, even old or extremely specific for a single use, may be useful for a long time. Not only for porting to some other, strange platform, but for issues like prior art in software patents, for getting inspiration for techniques or simply because you don’t know when it may be of use. For this reason, I try to create public access archives of source code I manage to get my hands on, especially when such codes may require a written license to acquire, but may then later be redistributed.

Up to now, I have prepared public archives of the following projects:

DOD OSCMIS: a very large web-based application (more than half a GB of code), created by the Defense Information Systems Agency of the US Department of Defense, and currently in use and supporting 16000 users (including some in critical areas of the world, like a tactical site in Iraq). I wrote a description here, and the source code was requested in writing during 2009. I am indebted to Richard Nelson, the real hero of such a great effort, for creating such a large scale release, that I hope will spur additional interest and contributions. I believe that I’m the only European licensee, up to now :-) The source code is available at the SourceForge mirror: http://sourceforge.net/projects/disa-oscimis/

NASA CODES: One of my oldest collection-and recovered by pure chance. Many years ago, we used to order CDs with source code on it (would you imagine it? How victorian…) since downloading them through our 14.4KBaud modems would have required too much time. So I ordered the Walnut Creek CD archive of the NASA COSMIC source code archive, a collection of public domain codes (mostly in Fortran) for things like “Aeroelastic Analysis for Rotorcraft in Flight or in a Wind Tunnel”. They are mostly obsolete, but since COSMIC was turned into a money-making enterprise that requires quite a substantial amount of money, I enjoy the idea of providing an access to the original codes.  The entire list of software descriptions is available here, and the codes are browsable at http://code.google.com/p/nasa-cosmic/source/browse/#svn/trunk.

Symbian: Ah, symbian. I already wrote about the high and lows of the Symbian OSS project, and since Nokia plans to shut down everything and make the source code accessible only through a direct request for an USB key or DVD, I though that an internet accessible archive would have been more… modern. It is a substantial, massive archive – I had to drop all Mercurial additions to make it fit in the space I had available, and still it amounts to 6.1Gb, Bzip-compressed. It is available at http://sourceforge.net/projects/symbiandump/files/.

I have performed no modifications or changes on the source code, and it remains under its original licenses. I hope that it may be useful for others, or at least become a nice historical artifact.

5 Comments

How to make yourself hated by academics.

I have been talking about OSS for a long, long time, and my first public conference on the subject is still imprinted in my mind. It was at a very important post-universitary Italian school, with a renowned economic department, and I got invited to deliver a speech about EU activities in support of OSS, to an audience mainly composed of academics from sociology, economics, political science and such. Just after my talk, one of the professors started a lively debate, claiming that I was a “crypto-communist, deluded and trying to spread the false model of the gift economy upon IT”. Heck, I stopped talking for a moment – something that the people that knows me would find surprising (I tend to talk a lot, on things that I like). I had to think about the best way to answer, and was surprised to find that most of the audience shared the same belief. One professor mentioned that basic economic laws make the very idea of OSS impossible, or only a temporary step towards a market readjustment, and so on.

Guess what? They were wrong. And not wrong a little – wrong a lot (but it took me a few years to demonstrate it).

And so, after all these years, I still find sometimes academics that improvise on the subject, claiming certainty of their models; models that, usually, include hidden assumptions that are more myth and folklore than science. Thankfully for the many ones that are not subject to this faults (Dirk Riehle comes to mind, as Rishab Gosh, Paul David, Francesco Rullani, Cristina Rossi, and many others) we have real data to present and show. I still sometimes open my talks with a mention from “Government policy toward open source software”, a book from AEI-Brookings where Evans claims that “The GPL effectively prevents profit-making firms from using any of the code since all derivative products must also be distributed under the GPL license”. Go tell that to RedHat.

Now, I have a new contender for inclusion in my slides; an article from Sebastian von Engelhardt and Stephen M. Maurer, that you can find in all its glory here. I will try to dissect some of the claims that are hidden in the paper, and that for example push the authors towards “imposing a fixed, lump-sum tax on OS firms and using the proceeds to subsidize their [proprietary software] competitors”. I think that Microsoft would love that – a tax on RedHat, Google, IBM! What can be more glorious than that?

I will pinpoint some of the most evident problems:

  • “For this reason, the emergence of fundamentally new, “open source” (OS) methods for producing software in the 1990s surprised and delighted observers.” Actually, as I wrote for example here, the tradition of collaborative development of software far predates Stallman and Raymond, and was the norm along with the creation of “user” (more appropriately “developer”) groups like SHARE (Society to Help Avoid Redundant Efforts, founded in 1955 and centered on IBM systems) and DECUS (for Digital Equipment computers and later for HP systems), both still alive. Code was also commonly shared in academic journals, like the famous “Algorithms” column of the “Communications of the ACM” journal. It was the emergence of the shrinkwrapped software market in the eighties that changed this approach, and introduced the “closed” approach, where only the software firm produces software. This is actually an illusion: in Europe, the market for shrinkwrapped software is only 19% of the total software+services marker, with own-developed software at 29%. We will return upon this number later.
  • “This made it natural to ask whether OS could drastically improve welfare compared to CS. At first, this was only an intuition. Early explanations of OS were either ad hoc (“altruism”) or downright mysterious (e.g. a post-modern “gift economy”). [Raymond 1999] Absent a clear model of OS, no one could really be certain how much software the new incentive could deliver, let alone whether social welfare would best be served by OS, CS, or some mix of the two.” Argh. I understand the fact that my papers are not that famous, but there are several excellent works that show that OSS is about the economics of production, and not politics, ideology or “gif economies”.
  • “economists showed that real world OS collaborations rely on many different incentives such as education, signaling, and reputation.” See? No economic incentives. People collaborate to show their prowess, or improve their education. Actually, this applies only to half of the OSS population, since the other half is paid to work on OSS – something that the article totally ignores.
  • “We model the choice between OS and CS as a two-stage game. In Stage 1, profit-maximizing firms decide between joining an OS collaboration or writing CS code for their own use. In Stage 2 they develop a complementary product, for example a DVD player or computer game, whose performance depends on the code. The firms then sell the bundled products in markets that include one or more competitors.” So, they are describing either a R&D sharing effort or an Open Core model (it is not well explained). They are simply ignoring every other possible model, something that I have already covered in detail in the past. They also ignore the idea that a company may contribute to OSS for their own internal product, not for selling it; something that is in itself much bigger than the market for shrinkwrapped software (remember the 29% mentioned before?) and that is totally forgotten in the later discussion on welfare.
  • “OS only realizes the full promise of cost-sharing when CS firms are present”. This is of course false: R&D sharing is always present every time there is a cooperation across a source base. But the article mentions only a simplistic model that assumes a OS company and a proprietary company (they insist in calling it Commercial Software, which is not).

There is a large, underlying assumptions: that OSS is produced now only by companies that create Open Core-like products. The reality is that this is not true (something that was for example found in the last CAOS report from the excellent Matthew Aslett) and the exclusion of users-developers makes any model that tries to extract welfare totally unreliable.

Ahh, I feel better. Now I have another university where I will never be invited :-)

, ,

3 Comments