Virtualization

From GO Wiki
Revision as of 12:54, 18 September 2010 by Sjcarbon (talk | contribs)
Jump to navigation Jump to search

Overview

This document describes the rational, high-level software choices, and possible problems of a move to a virtualized infrastructure.

Updates on tests and experimental implementations of a virtualized infrastructure at BBOP can be found here: Virtualization_progress

Rational

In addition to the general advantages of virtualization in a computing infrastructure, there are potentially several advantages especially important for GO.

Consistency of Development Platform

As an example, AmiGO's environments (production, development, 3rd-party development) are very distinct--different operating systems, package managment, and maintenance cycles. This can make agreement on packages and versions not only hard to discover, but also hard to maintain due to dependencies that may not be directly related to software development.

Using a shared VM image eliminates the above problems, giving a single unified development platform. In a less intensive form, a VM would allow there to be a common reference platform that development had to be tested against before being accepted.

In addition, especially with programmers at different locations, virtualization could help deal with hard-to-debug problems with by literally sending a copy of the current machine to somebody who might understand the issues better.

GO Software Distribution

VM images are potentially a great way to distribute GO software. Either as part of a larger infrastructure or from the command line on an individual machine, getting a VM running on a machine greatly simplifies the installation process of complicated software packages (similar to a bootable ISO image).

Also, with the flexible size of a VM image (compared to around 5GB max for a DVD image), it would be possible to add live databases and large data files directly into the image for easy use. For researchers who might not have local software support to help in the setup and maintenance of complicated packages and dependencies, it could reduce the amount of time spent on non-productive activities.

Speed to Production

A bonus to having this as a software distribution method is that it could be used internally to speed the release cycle of server-based GO software (e.g. AmiGO, Solr and data files). A small set of changes could convert a development image into a production image, which could then be immediately deployed. If a new production image was problematic, it could be immediately switched with an older one, causing minimal disruption to users.

Redundancy and Recovery

Given that no infrastructure is bullet-proof.

  • The ability to take a running image and move it to a different machine or facility with minimal downtime is a great feature.
  • The ability to rollback great swaths of infrastructure to known working point in the past can help to deal with unexpected problems in a way that does not affect end users.

Cloud Computing (buzzword!)

Although we do not currently have plans to implement any infrastructure on the cloud (permanently or on-demand), it seems like a good option to keep open; for example, it could be useful as an emergency backup or to create regional mirrors of data.

Platforms

In addition to the software mentioned below (what we currently believe to be the best solution for what we want), we have tried: Open Nebula, libvirt, XenServer, CentOS, Debian, and openSUSE.

Below is a brief discussion of the current experimental list and some explanation.

Ubuntu LTS

The preferred platform is Ubuntu 10.04 (two paragraphs of reasons cut out here). In particular, Ubuntu has tight integration and support for the virtualization infrastructure we currently prefer.

UEC/Eucalyptus

This is the virtualization infrastructure mainly supported by Ubuntu. While there are some annoying hooks (i.e. Canonical's Landscape service), it seems to be more community driven and open than the Citrix (Xen) equivalent and more usable than Open Nebula.

It supplies both high-level VM and storage interfaces and supports a fairly broad range of underlying technologies. It should also be possible to integrate with remote infrastructure.

Amazon API

Eucalyptus also integrates well with Amazon's "cloud" APIs (EC2 and S3), allowing some fuzzing between our private infrastructure and anybody who implements Amazon's computing and storage APIs (Amazon, other S3 providers).

KVM

KVM seems to be better supported by the Linux kernel and is the preferred method of virtualization for Ubuntu.

Caveats

While the cons list would seems to be fairly minimal with virtualization, there are several points which need to be considered.

Initial infrastructure cost

While most newer machines support virtualization in the hardware (necessary to get to make virtualization worthwhile), it is not worthwhile on older hardware or, given KVM's current connection to x86x architectures, non-x86x systems.

In addition, moving large images around can me a slow and time-consuming process. Either patience or switching to faster networking hardware (where not already in place) would be necessary to get all of the benefits.

Both of these issues are at least partially addressed by the fact that new infrastructure rollout is necessary anyways as machines are replaced in the normal course of things. If a virtualization infrastructure is planned and rolled out over time, additional cost could be minimal.

Increased complexity

Even though virtualization gives better management and resource allocation at a high level, the fact is that there is at least one additional layer between software and hardware, and management and resource allocation at a low level becomes much more complicated, albeit largely hidden from the administrator.

This can make troubleshooting of some kinds of problems more difficult and increases the number of ways that things can go wrong (also connected to para-virtualization versus full-virtualization).

Some of this is inevitable--computing has always gotten more complex over time as higher-level abstractions are created over old ones. The increased flexibility presented by virtualization will hopefully pay for the increased complexity and problems associated with it.

Inexperience

Related to the above, as with any new way of doing things, there is going to be a learning period where the solutions to new problems are slow and non-optimal. Hopefully, by using a well-supported software infrastructure with a large community around it, this will be kept to a minimum.

Monoculture

Put all your eggs in the one basket and--WATCH THAT BASKET.
— Pudd'nhead Wilson's Calendar

Currently, AmiGO is developed on one platform, put into production on two others, with software developed for on a fairly wide variety. Just the act of having AmiGO function in all of these different places helps maintain a clean architecture and good coding practices (e.g. bugs that are not apparent on one platform cause crashes on another). To some extent, it becomes tradeoff between robustness and time to develop and get into production. Given resource limitations, favoring the latter is probably the best at this time.

On a more paranoid angle, monoculture also increases the risk of a bug or security hole on one instance being trivially exploitable on all instances. I'm unaware of any specific attack against AmiGO/GO software, (only general attacks again, say, LBL), but the potential is there.