SlapOS Desgin: Technical & Economic

SlapOS Rationale

The goal of this document is to outline the economic rationale which drives SlapOS research and development.

Starting in 2007 many companies whose business is to sell open source software integration services have migrated their groupware from open source platforms to proprietary cloud platforms. Industrial companies that had been using proprietary groupware for many years have also opted for a similar strategy towards the cloud. Costs seem to be the driving motivation for change.

This presentation will analyse the cost of different approaches (on premise vs. cloud) and later introduce a list of economic requirements for an open source/Free Software cloud solution provider to meet in order to provide a competitive advantage over proprietary cloud providers.

Table of Content

  • Cost Comparison
  • Implications
  • SlapOS Goals
 

Cost Comparison

This section introduces scenarios of groupware implementation in a company using open source and proprietary software. It will also review the different commercial offers which are proposed to companies to implement similar solutions in the cloud.

Open Source Software on Premise

  • Open Source Software: 0€
  • Server: 2 * 1000 / 100 / 24 = 0.8€
  • System Administrator: 800 / 100 = 8€
  • Total: 8.8€ / user / month

Until recently (at the time of writing), one of the most cost efficient ways to implement groupware in a company was to select an open source groupware suite such as Horde, OBM, Group Office, Feng Office, et al., purchase two servers (so that one could replace the other in case of a hardware failure) and ask a system administrator to come one day per month to make sure everything is working well on both servers. This approach is quite typical of small companies.

A typical server for 100 users costs about 1000€ and is used in production for 2 years or 24 months. This represents a cost of 0.8€ per user and month. A system adminstrator costs 800 € per day and operates on the server once a month on average to create accounts, make sure there is enough disk space, install system security upgrades, etc. System administration represents a cost of 8€ per month and user. Overall, the total cost is 8.8€ per user and per month

By hosting more users on the server (ex. 1000) and using lower cost labour (ex. 200 € per day), the total cost per user can be reduced to less than 0.2€ per month. Therefore, what would cost 8€ in a small company in a country with high labour costs, could cost 0.2€ in a country with lower labour costs and in medium sized businesses.

Proprietary software on Premise

  • Proprietary Software: 20€
  • Server: 2 * 1000 / 100.0 / 24 = 0.8€
  • System Administration: 800 / 100 = 8€
  • Total: 28.8€ / user / month

Large organizations used to rely on proprietary groupware such as Lotus Notes. Some of these such as Valeo later migrated to cloud. The cost of traditional groupware implementation based on proprietary software is similar to the cost of open source with the additional cost for licensing the software. This can include not only the groupware application itself but also a complex system environment made of LDAP directories, proprietary operating systems, proprietary monitoring tools, proprietary office suite and proprietary asset management.

Typically, annual cost for the groupware environment licenses should be at least 240€ per user, resulting in 20€ monthly cost.

The total cost per user per month of the groupware environment is thus close to 30€. Increasing the number of users or reducing labour cost has little impact on the total cost since most costs are licenses.

Open Source Software - Cloud Based (XMail)

  • Advertising: 0.5€
  • Hard Disk: 100 * 7 / 1000.0 / 24 = 0.03€
  • System Administration: 100 * 10000 / 10000000 = 0.1€
  • R&D: 20 * 10000 / 10000000 = 0.02€
  • Total: 0.15€ / user / month

Many corporations have opted for free groupware once it became available on the cloud. It is however quite difficult to estimate the actual borne cost for such solutions since they are free. However, as solutions are commonly sponsored through advertising, a measure of the cost can be estimated through the number of clicks on advertising links made by each user every month. Another measure can be estimated based on the use of resources required to provide the cloud service.

Assuming a typical user tends to click a commercial link once per month and that commercial links are priced between 0.1€ and 1€ with a typical price of 0.5€ a low estimate of the "price" of free cloud groupware can be obtained.

Another estimation is based on the usage of physical and human sources required to produce the cloud service. The hardware cost of a service depends mostly on the cost of storage rather than on the cost of CPU. With a typical cost of 100€ for 1 terabyte of storage on a hard disk and an average of 7GB allocated per user, one can reach a hardware cost of 0.03€ per month. This calculation takes into account that storage is duplicated (ie. 14GB are needed to provided 7GB of redundant storage). It also takes into account that some users use all the 7GB storage while others mostly nothing, with an average usage of 3.5GB. The cost of CPU or RAM is considered to be close to zero as most users are only connected during small part of the day. Memory is thus shared efficiently by using a kind of multi-tenant architecture.

Intangible costs per user depend on the number of real users, since only those users are likely to provide advertising turnover through commercial links and can be considered as equivalent to users of a traditional approach.

Free cloud services usually have many subscribers and much less actual users, especially simultaneous users. The number of real users is a figure which is difficult to determine but should be considerably lower than figures published by marketing departments. One approximation could be the number of registered users on facebook (2012: 1 billion) and a 1% ratio for deriving actively engaged users to be 10,000,000. This 1% ratio is quite typical of Web 2.0 services as users subscribe to them more often than they actually use them. Still, the 1% ratio is probably overestimated as many free services actually show a 1:1000 ratio between real users and subscribed users.

If system administration is handled by a team of 100 engineers, the cost of system administration for 10,000,000 users is equal to 0.1€ per user per month. If the R&D staff consists of a small team of typically 20 people, R&D costs are equal to 0.02€ per user per month. The cost of free cloud groupware is thus equal to about 0.15€ per user per month.

Through various approximations, a monthly cost per user of 0.1€ to 0.5€ is taken into the calculation. This cost does not directly translate into a price, because a free cloud is financed by advertising. However, it provides an estimate target value which any open source cloud solution should match to be competitive in the long run, or which any open source cloud solution should match to be compatible with a business model based on advertising.

Proprietary Software - Cloud Based (XMail)

  • Google Apps Premier: 5€
  • OVH MS Exchange: 3.3€
  • Group Office: 3€
  • Total: 3-5€ / user / month

Commercial cloud groupware offers also exist and are much easier to analyse as their prices are usually public.

Google provides a "Premier" version of Google Apps at a cost of 5€ per user and month. OVH provides hosted Microsoft Exchange at a cost of 3.3€ per user and month. Group Office is a hosted version of the open source Group Office groupware. It is interesting to note that the price levels of commercial cloud offers are 30 times higher that their costs as we calculated them previously. There is thus still a lot of room for competition.

Price Comparison

  • Open Source - on premise: 0.2-8.8€
  • Proprietary - on premise: 28.8€
  • Open Source - cloud (Google Apps): 0.1-0.5€
  • Open Source - cloud (Premiere): 5€
  • Open Source - cloud (OVH MS Exchange): 3.3€
  • Proprietary - cloud (Group Office): 3€

By comparing different approaches to implement groupware in a 100 people company in a developped country, it was found that cloud-based solutions offer price levels which are 60 times lower than traditional solutions (0.1€ to 0.5€ vs. 28.8€). Paid cloud solutions lower prices by a magnitude of 10 compared to traditional proprietary solutions. This explains why many companies (Jaguar, Rover, BBVA) abandon traditional implementation of groupware and migrate to the cloud, especially in countries with high labour.

However, the level of profits generated by current paid cloud offerings is extremely high. Lower cost cloud offers are likely to reach the market very soon.

Traditional open source approaches may also remain a competitive option in countries with low labour costs and in companies with more than 1000 users. Economies of scale and mature open source software packages can reduce the cost of open source groupware to 0.2€ per month and per user in some cases.

Implications

In this section, the impact of cloud solutions for IT jobs are analysed as well as the hidden costs of (both traditional) and cloud IT which are often not taken into consideration.

Job Impact

  • Cloud: 10000000 / 100 = 100000
  • On Premise: 20 * 100 = 2000
  • Productivity increase: 50x

Per 2012, the cloud will have a major impact on the IT job market by reducing the number of open positions for system administrators and consultants.

A team of 100 system administrators can manage 10 million groupware accounts, whereas in a traditional approach, a single consultant can serve 100 users by spending one day per month, which means for a month of 20 working days, 2000 users. This figure of 2000 users compares to 100,000 users per system administrator for cloud-based approaches.

Thus cloud-based services can improve IT productivity by a magnitude of 50. In other words, what used to require 100 system administrators in traditional IT, only requires 2 in a cloud system, leaving 98 system adminstrators to an unpredicatble destiny.

The cloud plays the same role here as ERP systems did for accounting, or robots are doing for industry or mechanisation for agriculture: it automates the work of certain groups, who then become less competitive than machines. This economic process has been depicted in the essay "The End of Work" by Jeremy Rifkin. With the cloud, it is probably the first time that a category of workers (IT engineers) produce tools which will create massive unemployment for the same category of workers (IT engineers).

Understanding the impact of the cloud on the IT job market is also a better way to define what "cloud computing" is: the automation through software of IT processes which used to be managed by humans. Currently, cloud computing focuses mostly on the automation of system administration. However, research in cloud computing shows that the process of tailoring ERPs for small companies can be automated, too and that the configuration of applications could also be done by software rather than by consultants.

Risk of Data Loss

An interesting aspect of cost comparison which is seldomly taken into account is the risk of data loss. Corporate data can be very valuable. Losing accounting data or customer data can lead a stable and profitable company to bankrupcy in a matter of months.

There are already a few examples of data loss on the cloud either for technical reasons (eg. "Sidekick story") or for social reasons (eg. "story of erased google account"). There are also numerous stories of companies that use Storage Area Network (SAN) device and backup solutions from reputable makers but still lost their data because they did not monitor the alarms of either system well enough. The SAN filesystems got currupted and the backup solution had no longer been running for several months.

Is the cloud safer or not than traditional IT from this point of view?

It is hard to tell without numbers. Some companies with perfect backup plans could bear less risk than the cloud. Many companies implement a poor backup plan and could reduce their risk by migrating to cloud.

One thing is certain however: current cloud centralized architectures are based on large data centers and are operated by the same company all over the world. Centralized architectures are quite weak in case of strike, critical bugs, virus or terrorist attack. It is not only a matter of lack of technical redundancy but also of lack of management redundancy.

Also, the perceived risk of data loss in cloud is becoming higher and higher. Just like with air transportation, which is the safest transportation system, fears of accidents can lead some users to reject the cloud. A Cloud disaster, which is already predicted to happen in 2012 could generate a lot of distrust for the cloud.

SlapOS Goals

This section defines SlapOS requirements by combining an architecture which reduces the risk of data loss and at the same time matches or surpasses the best cloud offers in terms of costs.

SlapOS Goals

  • Share application R&D
  • Share automation software
  • Share hardware capacity
  • Share data loss risk
  • Share management
  • Goal: <0.15€ / user / month including mitigating the risk of data loss.

The goal of SlapOS is to provide a competitive cloud solution which does better in reducing risks of data loss than any other solution.

A simple word summarizes the requirements to build such a solution: share.

Application R&D is shared by using open source applications such as Horde or Wordpress which are widely spread and maintained by communities. The cost of R&D is thus close to zero for end users and competes with the costs of R&D of companies with million users.

Automation software, the core of SlapOS, is based on existing open source tools maintained by communities such as Buildout or supervisord. This reduces the cost of development of automation tools by a magnitude of 10 or more. Other automation components are shared among all applications, through a concept of a stack which is itself open source. All LAMP (Linux, Apache, MariaDB, PHP) could for example share the same stack and benefit from improvements made to the stack, which provide enough freedom for each application to implement fine tuning.

There are already more than 100 LAMP applications in SlapOS. Adding one more application with complete automation of deployment, backup, resilience and front-end acceleration taks less than four hours.

Hardware capacity should be shared (and waste eliminated) by efficient system design and capacity sharing among providers.

A single 1000€ server should be able to host at least 1000 to 10,000 instances. Some form a multi-tenancy must thus be provided by default in SlapOS, since most open source applications are not multi-tenant. This form of multi-tenancy is called POSIX in SlapOS: it is an operating system which allows multiple processes with different users to share the system RAM, CPU and also the same application code through shared libraries. By using POSIX processes (rather than virtual machines), it is possible to implement dynamic allocation policies which can be split from a single server into 100 containers, each of which with a groupware instance of Horde and 10 users. By starting and stopping POSIX processes automatically, the number of containers can be increased to 200 or more. The cost of a complete groupware suite for a single company can be as low as 1€/month. This figure should be compared to approaches based on Amazon EC2 which price is 10 times higher if not 100 times. And since a kvm virtual machine can be a POSIX process, using POSIX processes as the common model for cloud services can cover more possibilities than cloud models based on virtual machines.

Another form of hardware capacity sharing consists of one cloud provider supplying available unused capacity to another cloud provider. This approach reduces the cost of hardware resources for every cloud provider since it optimizes their usage.

Risk of data loss should be reduced by sharing risk accross continents. Every SlapOS hosted application should be replicated in 2 or 3 regions in the world. In case of a failure of SlapOS cloud resources in one region (electricity, connectivity, natural disaster, etc.), another replica of the application can provide continued service.

SlapOS should be designed to resist management mistakes. Taking a wrong management decision on a single SlapOS node should not destroy the entire system. Management should thus be distributed and shared accross multiple organizations in the world, each running its own SlapOS cloud.

By meeting all requirements, SlapOS can compete in price with the lowest cost proprietary cloud yet provide higher resilience and data protection than any other solution.

Thank You

Image Nexedi Office
  • Nexedi SA
  • 147 Rue du Ballon
  • 59110 La Madeleine
  • France