What if I could use Vagrant to configure and create reproducible environments for development, test, and production? This is the question I thought to myself while working on the planned activity to setup a MongoDb cluster. After spending time testing the cluster with VirtualBox and Vagrant and then switching to knife-solo to bootstrap droplets on Digital Ocean, I opted to dig further into my question.
With the release of Vagrant 1.1 in mid-March, it was now possible to manage virtual machines backed by providers other than VirtualBox. HashiCorp, the creator of Vagrant, provided a compelling preview showcasing Vagrant managing Amazon EC2 servers. The timing was right to investigate the possibility of using Vagrant as the tool to define my environment and subsequently set it up on either VirtualBox or Digital Ocean. As such, I redirected my remaining few weeks for the first sprint to tackle this challenge.
Value & Exit Criteria
Chef had proven extremely useful for defining configuration for individual servers. Vagrant would compliment Chef by providing a common, consistent interface for managing an environment of servers. This would help further minimize system regressions and reduce the operational cost for managing multiple environments.
To claim success on the task, I defined the end state. As a developer and administrator, I wanted to run the following commands to stand up a six-server environment with VirtualBox or Digital Ocean respectively:
$ vagrant up --provider=[virtualbox|digital_ocean] --no-provision $ vagrant provision $ curl http://<proxy>/api/tasks
The environment had to be defined with a single
Vagrantfile creating a proxy
server, two Node.js application servers, and a MongoDb replica set. After
provisioning with a single Chef application cookbook, the system had to be
fully operational accepting requests.
In my previous job, we had used Vagrant to create reproducible development
environments for the development team. Beyond creating a basic
to spin up a machine provisioned with Puppet or Chef, my knowledge of Vagrant
was limited. As I dove into the internals of Vagrant to meet my objective, I
was pleasantly surprised to find an extensible plugin framework. The framework
supports new configuration, commands, virtual machine providers, and
action hooks. In fact, most of Vagrant’s built-in features are implemented
Operational use cases for creating and managing an environment could be implemented as a set of plugins. I looked to reuse existing ones where possible and created new ones when needed. The four plugins that helped achieve my objective are discussed below.
Digital Ocean Provider Plugin
To create, rebuild, and destroy Digital Ocean droplets, I needed a Vagrant provider plugin. Luckily, John Bender had already established a GitHub project that delivered the basic functionality. Unfortunately, it was missing support for SSH keys (it used the insecure Vagrant key) and relied on a root account for subsequent provisioning. John was gracious enough to support my modifications ultimately transitioning the project over to me. Adding features to this plugin was a great exercise in understanding the fundamentals of Vagrant and I highly recommend anyone interested to browse the source code.
As of today, the plugin provides the following features:
- create and destroy droplet instances
- rebuild a droplet instance while retaining its assigned IP address (custom command)
- power on and off a droplet instance
- setup a SSH public key for authentication
- create a new user account during droplet creation (allowing me to disable the root account during provisioning)
- provision a droplet with the shell or Chef
While this plugin met 80% of my objective, creating new droplet instances introduced a problem. An IP address is not assigned to a droplet until after it is created, however, my Chef recipes required knowledge of it upfront.
Host Manager Plugin
To solve this dilemma, I opted to reference servers in my Chef recipes using
host names and leverage a Vagrant plugin to synchronize a
/etc/hosts file across
the environment to resolve the names. I created a Vagrant 1.1 compliant
plugin, vagrant-hostmanager, that hooked into the
up action for a new
server. After server creation, a line containing the server’s new
IP address and host name is added to the
/etc/hosts file of each active
server within the Vagrant environment. Once the
vagrant up command completes,
all servers defined within the
Vagrantfile can resolve to one another using
Due to this approach, I had to disable provisioning when calling
That is why my end state included the
--no-provision switch and a second
command to provision the servers.
Now that servers within my Digital Ocean environment could resolve to one another, I could move forward with configuring a replica set. Before I had started my investigation into Vagrant, I had attempted to use the mongodb cookbook. Unfortunately, the cookbook required a Chef server to search for replica set members and I had opted for Chef solo a few weeks earlier. Additionally, it felt awkward that a recipe provisioning a single MongoDb replica set member would also attempt to initiate a replica set if all members were available.
A Vagrant plugin seemed to be a better fit – it had access to the
Vagrantfile containing configuration for all servers in the environment and
therefore could act on the environment itself. I created a plugin,
vagrant-mongodb, that provided an adminstrator with new configuration
options in the
Vagrantfile to describe a replica set. Here is a brief example:
1 2 3 4 5
With a replica set defined within a
Vagrantfile, the plugin will check
if all members are available, and if so, initiate it. By default,
this happens automatically after executing
vagrant provision, although this
behavior may be disabled and a custom command executed instead.
With the three plugins above, I could now meet my objective. However, I took
the time to clean up my use of Chef from earlier weeks. Since I was replacing
knife-solo with Vagrant, I had the opportunity to use
Berkshelf for cookbook dependency management. I collapsed my
marinara-kitchen project into a single application cookbook called
marinara-cookbook containing the recipes I wrote earlier. I defined my
cookbook’s dependencies in the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
With the Berkshelf Vagrant plugin installed, all cookbook dependencies are automatically available to each server during the provisioning process.
With plugins to manage Digital Ocean droplets, synchronize a
for server resolution, and initiate MongoDb replica sets, I was ready to
define my application’s environment in a single
Vagrantfile. Below is my
file that works with both VirtualBox and Digital Ocean (with a few manual
tweaks required until Vagrant 1.2 is released):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119
You will note the declaration of private networks and static IP addresses in the file. This enables the machines to communicate with one another when running locally using VirtualBox. Currently, I have to ignore the private IP addresses when using Digital Ocean.
When Vagrant 1.2 is released, a user will be able to configure
override attributes for a specific provider. In the file above, I can move
ignore_private_ip configuration into the Digital Ocean provider without
manually changing values.
More importantly, Vagrant 1.2 introduces the possibility of executing actions in parallel for a multi-server environment. This will cut down my current time to build a new six-server environment from 15 minutes to ~3 minutes.
Overall, I’m quite pleased with the outcome. With this sprint complete, I can now create and manage a multi-server environment for development, test, and production with the combination of Vagrant and Chef. Over the next six weeks I’ll be shifting my focus from infrastructure to product management defining my product’s scope and drafting a user experience in preparation for development. Stay tuned for more.