I had the great pleasure to converse on twitter along with the exceptional Simon Wardley, a longtime expert and researcher on company innovation, evolution and.. cloud computing. Among the limit of 140 characters, it was quite difficult to convey any sensible concept in such a short space. One of the interesting thing is that it is difficult to provide a sensible comparison between private and public clouds in absence of real numbers. So, since we just finished a research report for a major UK public authority, I will just add my own 2 eurocents and present a summary table of some examples of private, public and dedicated cluster costs:
System $/Core-hour Hopper  $0.018 Montero-Llorente  $0.04 Magellan (overall)  $0.04 Class 1 server/workstation  $0.046 Cornell RedCloud  $0.058 Our estimate $0.06 Amazon cc1.4xl, resv. instance $0.062 Amazon cc1.4xl $0.093 CINN  $0.1
This is of course just a snippet of more than 40 pages; cost includes management and amortization over 3 years for hardware, 5 years for infrastructure. Our own estimate is for a self-managed, self-assembled system with no best practices, while Magellan is a realistic estimate of a median cost for a well-managed and well-procured infrastructure. Hopper is a custom cluster out of commodity hardware and can be considered the best approachable price point for 2011 in terms of cost/core for a private cloud as well. In the paper (that I hope will be published soon) there will be additional details on the actual model, the estimates and the sources for the data. Hope it may be useful for someone.
(this is a repost of the original article with some corrections. Includes the Oxford TransferSummit 2011 presentation based on this data)
What is the real value that Open Source has brought to the economy? This is not a peregrine question. Since most of the current evaluation methods are based on assessing “sales”, that is direct monetization of OSS, we are currently missing from this view the large, mostly under-reported and underestimated aspect of open source use that is not “sold”, but for example is directly introduced through an internal work force, or in services, or embedded inside an infrastructure. Summary: OSS provide cost reduction and increases in efficiency of at least 116B€, 31% of the software and services market.
Getting this data is, however, not easy. There is an easy approach, called “substitution principle”, that basically tries to measure how much a collection of hard-to-measure assets is valued by counting the sum of the money necessary to substitute them; for example, counting the value of all the Apache web servers by adding the cost of changing them all with an average, marketed substitute. This approach does not work, for two reasons: first of all it introduces a whole world of uncertainty, given the fact that software is never perfectly exchangeable with an alternative. The second is related to the fact that users may be unwilling to pay for an alternative, so from that point of view the real value is much lower. This is, by the way, the (erroneous) way that RIAA and other rights organizations measure piracy losses: by counting how many times the copy of a film is downloaded, and assuming that all the people that downloaded it would have paid for a full cinema ticket if piracy did not exist. It is obviously wrong – and would be equally wrong if we applied the same principle.
Another approach is to measure the revenues of companies that are adopting an OSS-based business model, something that we have extensively studied in the context of the FLOSSMETRICS project. The problem with this approach is that it totally ignores the work that is performed without a monetary compensation, and it under-reports the software that is distributed widely from a single source (for example, the open source code that is embedded in phones or routers). A purely monetary measurement also ignores inherent improvements in value that can derive from an improved technology. Let’s make an example: let’s imagine that in the world, all television sets are black and white only, and only recently a new and improved television set can provide color. The new TV sets costs quite a lot more than the old B&W ones, so if we imagine that all the current TV viewers want to move to color the TV set provider can obtain a total amount of money that is the product of the cost of the new TV set multiplied by the number of viewers. The company is happy
Now, let’s imagine that a magic signal allows the old TV sets to show color images. The company that produces the color TV sets is clearly unhappy, since its value dropped instantly to zero, but on the other hand all the people with B&W TV sets is happy, even if there is no monetary transaction; the user value increased substantially. We need to capture this value as well, since a substantial amount of this economic value is hidden in the user balance sheets. This means that we need to find a different, alternative way to measure OSS value: enter macroeconomics!
We can start from the overall economic value of IT in general. There is one thing that we know for sure: the total economic value of a country or a region like Europe – 12.3T€ (trillion of Euro). We also know the average IT expenditure of companies and Public Administrations, that is 4% (source: Gartner IT key metrics data, EU eBusiness-Watch) with wide variations (small companies: around 7%, going up with size up to the average for Fortune 500: 3%). This means that the average IT spending, including services, employees, hardware, software, whatever. This means that the overall IT spending in Europe is approximately 492B€, of which 24% is hardware (source: Assinform, Gartner, IDC) – which means that software and services market is valued at 374B€. (Estimates from Forrester are in the same range, so we are at least consistent with the big analyst firms)
Still with me? Good! Now, the next step is estimating the savings that are directly imputable to open source. We have two sources: an internal source (code replaced by OSS) and external (savings reported by IT personnel through use of OSS). Let’s start with savings from OSS adoption, that can be estimated (using data from Infoworld and our data from COSPA) at 15% for “light” adopters (less than 25 OSS products used) to 29% for “heavy” adopters (more than 25 OSS products), up to the 75% of specific cases (reported by Gartner for maintenance and licensing). Taking into account the share of use of OSS in general and the variation in use of OSS among different sizes, we can estimate that the savings directly introduced by OSS amount to 41B€ – those do not appear anywhere but in the adopters balance sheets, that is in a reduction of IT expenses, or a better result for the same IT expenditure (think about the TV set example outlined before).
And now, software development. It may sound strange, but only a small part of software is ever developed for the market – what is called “shrinkwrapped”. The majority of software is developed (internally or through external companies) for a specific internal need, and is never turned into an external product. In fact, when we consider the “service” part of the non-hardware IT market, we discover that nearly half of that value is actually sponsored software development, and the remaining 35% is non-software services (support, training, ancillary activities). This means that in Europe, 244B€ are software spending in a form or the other (for example, employee wages).
What can we say about this software? We know that a part of it is Open Source, because the majority of developers (69%, according to Evans Data) is using open source components within their code. We also know, thanks to Veracode, that “sampling … find that between 30 and 70% of code submitted as Internally Developed is identifiably from third-parties, most often in the form of Open Source components and Commercial shared libraries and components”. In our own database, we found out that the role of commercial shared libraries is hugely dependent on application type and vertical sector, and it falls consistently between 15% and 30% of the code not developed from scratch. Using a very conservative model, we can thus estimate that 35% of the code that is developed overall is based on Open Source, and this means that there is both a saving (software that is reused without having to redevelop it) and a cost, introduced by the need for adaptation and the “volatility cost”- that is, the risk introduced by using something developed outside. Thankfully, we already have quite a lot of information about these costs, thanks to the effort of the software engineering community; some details can be found here for those that really, really want to be put to sleep.
Applying the software engineering costs detailed in my previous article (volatility, increased cost for code re-factoring, glue code development) we can estimate that the savings introduced by OSS are, in a very conservative way, 31% of the software-related part of the IT ecosystem, that is 75B€. The real value is higher, mainly because reused OSS code tends to be of higher quality when compared with equivalent proprietary code (data and academic references available here) but I will leave this kind of evaluation for a future article. We can however say, with quite a good level of certainty, that the lower bound of savings that OSS does bring to the European economy is at least 116B€ - the majority of which does not appear in the “market” and only in a minimal part in the balance sheets of OSS companies (consider that only Red Hat is now approaching 1B$ in revenues). It is savings and increased efficiency of companies and Administrations that use OSS, something that was already discovered: “Finally, comparing the individual data on firms with turnover of less than 500,000 euros with the variable on size classes of customers (by number of employees), one can hipotesize a correlation between the use of software Open Source and the ability to attract customers of relatively larger scale. At the same turnover, in other words, companies “Open Source only” seem to have more chances to obtain work orders from companies with more than 50 employees (ie medium – large compared to our universe of reference).” (source: Venice study on Open Source) or the fact that revenue-per-employee ratio is higher in companies that adopt open source software (on average by industry, OSS-using companies have a revenue-per-employee that is 221% of the non-OSS controls). It is also important to recognize that this is only a measure of the direct impact of OSS. The reality is that software has a substantial impact on revenues; an example I found out is Siemens, that with 70B€ in revenues spends 5% in software drives 50% of its revenues. A similar impact can be expected of the savings introduced by OSS – something that we will talk about in a future post.
We were particularly happy about out work on EveryDesk – a portable, fully working live Linux installation on a USB disk. But we found out that more and more people were looking for more space, a more modern environment, and in general to refresh things. We have been busy with out other pet project – CloudWeavers, a private cloud toolkit, and we redesigned EveryDesk to be the ideal client environment for companies and administrations that are moving totally or partially to a private or public cloud. We took several ideas from ChromeOS, but frankly speaking the hardware support was extremely limited, and even with exceptional ports like Hexxeh’s “Lime” the user experience is still less than optimal. We have basically redesigned everything – the base operating system is now derived from OpenSuse (mainly thanks to the excellent package management tool, that drastically increases the probability that the system would continue to work after an update – a welcome change from Ubuntu), we integrate Gnome 3, the latest Firefox and Chromium on a BTRFS install that supports compression and error concealment, so it works properly even on low-cost USB devices. On an 8Gb USB key, you get 4Gb free, and all the apps at your disposal, ready to go.
The only major change in hardware support is the fact that EveryDesk is now a 64-bit only operating system, but we believe that despite the limitation it can still be useful at large. It integrates some components that are maybe less interesting for individual use – for example the XtreemFS file system, that can be used to turn individual PCs into scale-out storage servers in a totally transparent way, and with great performance, or many virtualization enhancements. On the user side, we already installed some of our favorite additions among fonts, software, and tools; Firefox uses by default the exceptional Pdf.js embedded viewer, that uses no separate plugins and is faster than Adobe Acrobat, and there is the usual assortment of media codecs and ancillary little things.
We love every moment that we work on this project, and I would like to thank the many people that helped us, sent criticisms and praises. One wrote “I can’t believe how well it works, without time lags I normally associate with running on a CD or a thumb” and I can’t thank our users enough – they are our real value. As usual, you can download EveryDesk from Sourceforge.