Three Predictions about Cloud Computing for 2011
With all the talk in 2010 about cloud computing you’d think the entire Internet was running on it. We’re at the point now with cloud computing as we were in the late ’80s through the mid-late ’90s with networking. Everyone can clearly see the benefits in cloud but the market is hyper fragmented as different pockets of users form a community around one of the available solutions. To ensure new readers are aware, while I am employed by Rackspace Hosting working on the OpenStack project, the opinions expressed on the blog are mine and I try to present a non-biased view of the market.
With that opening I’ll dive into the three items of importance to cloud computing for the coming year..
..with additional more minor predictions in italics.
1. Cloud computing needs will change as we move from early adopters to mainstream users
Thus far the primary users of cloud infrastructure as a service (IaaS) offerings have been early adopter technology savvy users. Those users may be founders of a Web 2.0 startup, consultants working for the R&D department of a system integrator, or forward thinking IT professionals on enterprise IT strategy teams. Based on the stats in the chart, we still have less than 2% of the top 500k websites hosted on IaaS. In 2011 this number will grow to 5-10% of the top 500k sites, more than doubling again like it did in 2010.
Source: http://www.jackofallclouds.com/ - Guy Rosen
Inside startup communities everyone will be using the cloud to start their business and in enterprises departmental innovation success stories will start to bubble up to corporate leadership. This doesn’t mean existing applications will be migrated — people will experiment with migration as disaster recovery innovation but it won’t be a major driver of cloud growth in 2011.
The major driver of growth will be new applications and much of this growth won’t be consumer Internet sites that are easy to track making the “who’s winning in cloud” leader board more difficult to display. This next wave of adopters will also require additional levels of support as they won’t have the same “DIY” mentality as early adopters. Cloud providers will need to raise their own service levels or spend significant effort building a system integrator and consulting ecosystem that can provide it for them.
The ecosystem of tools built on the IaaS cloud APIs will be a foundation to enable the higher service levels. They will be utilized by the practices for the SIs and consultants as well as the software development teams of many ISVs. For cloud providers that do not yet have an ecosystem built around their API this will be the year they move on to adopting one of the open APIs with market traction. Enough providers will use a group of 3-5 APIs that ISVs and startups will refuse to developer for others. API abstractions like jclouds, Libcloud, and Deltacloud will start to add depth rather than additional breadth.
2. Technology heavyweights with large developer communities will escalate their efforts to define and control PaaS
2006 brought us Rackspace Cloud Sites (originally branded as Mosso)…
2007 gave us Force.com…
2008 launched Microsoft Windows Azure and Google App Engine…
2009 delivered Heroku’s commercial release and moved VMware into the platform space with the acquisition of SpringSource…
2010 veterans like Red Hat and Oracle [PDF] start announcing platform strategies and making acquisitions such as the recent purchase of Makara (by Red Hat)…
Over the past few years many platforms and application frameworks simplified development by providing a foundation and abstracting away lower level details. This came with some drawbacks as most frameworks were not aware of their resource utilization nor did they have the ability to utilize the programmatic capabilities of IaaS to change their resource allocation based on load. In 2011 many platform solutions will become tightly integrated with IaaS APIs providing dynamic resource management — the auto-scaling cloud workload across public, community, and private cloud installations will become an early adopter reality.
Building a PaaS solution is possible for a startup if they make it compatible with widely accepted development languages and frameworks as Mosso did by selecting PHP/Python/.NET with support for common applications like WordPress, Drupal, Django, and more. Another example is what Heroku did with Ruby and PostgreSQL. The heavyweights are the only players with deep enough pockets and the required patience to push new programming dynamics. Microsoft will look at how things are evolving and in order to defend .NET/C# will embrace the Mono Project creating a real threat to Java in the enterprise.
Java won’t take things laying down. Despite the fallout in 2010 over Oracle vs. Google, over the Apache Foundation first joking with Oracle and then stepping down from the JCP Executive Committee, and the many other examples such as James Gosling, the creator of Java, coming out against its new steward Java still maintains #1 position on the TIOBE index. Oracle, IBM, and VMware all have deep pockets and big revenue streams tied to the continued success of the language. IBM, while late to the PaaS party, will tie together Tivoli, WAS, and other components to build a robust platform for their customer base.
The blog-o-sphere will erupt in debate about “what is a true platform as a service” as much as they went on and on in 2010 about infrastructure as a service APIs. Despite what the pundits believe the majority of the enterprise IT dollars will go towards “false platform” private cloud solutions. Making the leap for major projects from current development methodologies and procedures to one of the new platforms will be too much — IT organizations need evolution, not revolution.
3. Enterprises will begin to evolve their virtualization deployments into private clouds and they’ll expect networking and audit controls beyond the capabilities of many current systems
Cloud deployments in enterprises will go in two different directions depending on the expectations of the project sponsoring executive team. Departmental usage of cloud that flies under the official process RADAR is not what I’m talking about. Over the past year I’ve had conversations with numerous people on Fortune 500 IT strategy teams and cloud is being looked at a couple of different ways. One group looks at it only as a technology solution that will magically make their operations more efficient. Another group realizes that the automation cloud brings only benefits them if process changes happen in parallel with with the systems improvements. Enterprise cloud projects that are only technology focused will not provide any meaningful savings and enterprises going this route will become disenfranchised. Virtualization was about consolidation not automation and because of that it didn’t include business process changes that cloud requires. You can’t simply install a “cloud upgrade” to your virtualization system and instantly have a cloud.
The cloud projects are going to run into a second set of hurdles. Up to this point typically departmental non-audited, non-regulated applications have been deployed by enterprises on clouds. In 2011 projects will need to address the corporate risk management and IT audit requirements. Major public clouds such as Amazon Web Services and Rackspace have addressed and received various attestations such as SAS70 Type II (Rackspace example), ISO27001 (AWS example), and PCI DSS [PDF] (Visa Global List of Validated Service Providers) showing that it is possible to build cloud services that meet compliance requirements. For enterprise projects to be successful they need to involve risk and audit up front so the proper control mechanisms are in the deployment. Because cloud is about workflow automation having to insert a manual audit control late in the project in order to meet the launch plans will eliminate many, if not all, of the projected benefits.
Corporate risk and IT audit teams will invalidate a number of cloud software fabrics and those platforms will quickly try to re-engage on projects by announcing partnerships with security companies. Platforms that come from service provider and government backgrounds such as OpenStack have a head start along with platforms that have evolved from enterprise DNA such as VMware. As enterprises start to spend significant dollars on cloud the R&D investment in cloud platforms will dwarf what has been spent to date and any head start present currently can see it easily vanish in 2011.
Audit controls for cloud platforms need to include both host and network services. It is possible to architect a cloud and map the controls into existing systems though this won’t be instantly turn-key or easy in 2011. Network virtualization will make cloud systems more flexible at a cost of making compliance controls more complicated. Process wise this also introduces another department into the cloud deployment discussions. Most clouds deployed in 2011 will focus on server automation and networking will be addressed in subsequent phases of the technology transition in 2012+.
Conclusion
2011 will be another big year for the adoption of cloud technology but these fundamental shifts happen slowly — especially when they involve people learning new processes and not just transparent technology replacement. When done right, cloud will make IT vastly more efficient and when cost of services decline the demand for those services often skyrocket.
This post focused on the cloud computing. I’ll be making other posts in January about distributed storage platforms (aka. “cloud storage”) and why they’ll be important for enterprises to understand and have readily available to their users before the middle of the decade. It is a fundamentally different problem as many cloud storage systems can be installed transparently without end user process changes.
How to tell the difference between “cloud” and “virtualization”
Many people seem to think “cloud” is just off-premise “virtualization”. Cloud comes in a few flavors and I’ll argue that you can have “private cloud” either hosted off-premise in a provider’s facility or in your own. The fundamental difference between cloud and virtualization is the goal of cloud is to automate provisioning (this applies to IaaS, PaaS, and SaaS) and the goal of virtualization is resource utilization optimization. You can (and many providers do) use virtualization as the basis for building a cloud but it is not required.
If we take a look at the Reductive Labs presentation from OpsCamp slide 3 illustrates the primary benefit of cloud. Cloud helps companies even if their minimum unit of work is larger than a single host machine where virtualization just adds overhead in that case. The difference between “cloud” and “grid computing” or HPC is that grid/HPC process jobs in a batch manner rather than serve interactive applications. You can build a compute grid on top of a cloud but not vice versa.
Other folks are saying “private clouds can’t exist because you can’t have rapid elasticity and pay for what you use”. For a small company you may not be able to have a private cloud but for a large enterprise with many business units you certainly can. An IT infrastructure BU can provide other organizations in the company all of the requirements of a cloud.
For public cloud to succeed they need to provide all three
Depending on the current utilization across an enterprises infrastructure they may be able to defer spending for a number of years by moving to a fully cloud enabled business. Right now many departments cling to servers they don’t need because they’re afraid if they release it they’ll never get it back. With cloud removing that fear resource hoarding ends and many enterprises will have a significant increase in available computing power.
Over the long term if the public computing clouds continue to grow, increase their transparency, and optimize their delivery models it will no longer make financial sense for enterprises to build their own infrastructure. Public cloud providers will need to prove over the next decade they can deliver on all three corners of the “impossible triangle”.
Cloud Computing, “For Everyone, Not Everything”
Cloud computing is a broad term that covers Internet based services that provide SaaS (Software as a service), PaaS (Platform as a service), and IaaS (Infrastructure as a service). SaaS services are the most commonly used cloud solutions — web based e-mail is the prime example. The most widely used PaaS offering is probably WordPress.org unless you consider customizing your Facebook profile a very restricted PaaS. IaaS is the newest of the cloud services with the most well known example of Amazon Web Services which includes EC2 (cloud servers) and S3 (cloud storage).
Until Hotmail launched in 1996 we all pretty much had an e-mail client on our own system and potentially had to run our own mail server if we didn’t want to have a mailbox tied to our college or ISP — now almost all of us use any number of SaaS e-mail services. Many of these e-mail services now include full features that businesses expect such as Rackspace E-mail or Google Apps Enterprise.
Before cloud based services if you wanted to have a website you had to run your own server until GeoCities launched in late 1995 — now PaaS providers from GoDaddy, for low price, to Mosso, for horizontal scale, provide very capable platforms to deploy a website without having your own server.
Now IaaS providers like Amazon, Terremark, and Rackspace are eliminating the need to always deploy and manage dedicated configurations for complex applications. Before these type of IaaS offerings companies like Twitter would end up with their own datacenters and dedicated infrastructure. Load testing services from companies like SOASTA would be cost prohibitive to offer.
So what about the title, “For everyone, not everything”? It sounds like cloud has the capability to do everything now doesn’t it? In a broad sense, yes, it can do a bit of everything but specific use cases in all service times aren’t a fit for cloud. In the e-mail world if you want to do offline messaging on an airplane you want a mail client. At the platform service level perhaps your application runs 10x faster if you can customize a couple of libraries or it just doesn’t work at all without those changes. The infrastructure offerings force you to re-architect for horizontal over vertical scale to use them effectively.
Many other use cases aren’t a fit for the cloud yet. Take video rendering as an example; it is much less expensive to buy a video card capable of performing rendering than it is to stream the rendered video over a network as 30 JPGs per second. Another example is a retail POS system, at least some of the functionality needs to be in the store — you don’t want to stop selling things if network connectivity is lost. Many more explanatory and reasonable examples abound.
Will cloud ever be the answer for all computing needs? I doubt it, but over time it will be used to solve more problems because a centrally managed pool of resources provides greater efficiency and flexibility. An example on this is utility power; we use it almost exclusively now but for a few use cases we still need generators. Cloud will succeed and it will be adopted for a wider set of use cases over time as it will address those use cases better than previous generation solutions.
