Cloud Foundry on AWS - Part 1: Deploying a MicroBOSH

2 Jul, 2013

Having spent some time experimenting with Cloud Foundry using Pivotal's flagship deployment at run.pivotal.io, I wanted to see how easy it was to deploy Cloud Foundry myself.

The recommended deployment mechanism for Cloud Foundry uses BOSH, "an open source tool chain for release engineering, deployment, and lifecycle management of large-scale distributed services". Once it's up and running, Cloud Foundry actually has no dependencies on BOSH, and in fact, it's technically possible to deploy CF by other means. But BOSH makes it easier to manage an existing Cloud Foundry deployment, so we'll stick with that.

(A word of warning: the official documentation for "Deploying Cloud Foundry on AWS" currently describes a installation process involving the bootstrap-cf-plugin gem. I recommend avoiding that path, as it conflates installation of BOSH and Cloud Foundry in a confusing way. Let's stay focused on BOSH, for now.)

BOSH is itself a distributed system, and can be deployed across multiple nodes (e.g. EC2 instances) for better performance and resilience. For our purposes, though, it's sufficient to have all the BOSH components installed on a single node. Such a single-node deployment is called a "MicroBOSH".

The BOSH project distributes MicroBOSH as stemcells; where "stemcell" is their term for a raw machine image. They provide an AWS-specific stemcells, which can be used to build AMIs, as well as stemcells for other target infrastructures (like vSphere and OpenStack).

Before you get too excited, though, you'll need an "inception server". Despite the fancy name, this is nothing more than an EC2 instance in your target AWS region, that you can SSH into, to use as a staging point for MicroBOSH installation. It's required because:

Any Linux instance running Ruby 1.9.x should do. If you don't have one handy, you can use one of:

Once you have an inception server running, SSH on in.

Option A: bosh-bootstrap

At this point, you're welcome to take a shortcut by way of Dr Nic's bosh-bootstrap project, which mostly automates the remainder of the process.

Option B: DIY

After some initial experiments with bosh-bootstrap, I wanted to understand more about what it was doing, so ended up building my MicroBOSH manually, using bosh-bootstrap as a guideline. If you're a sucker for punishment, like me, then read on ...

You'll need the BOSH command-line toolset. Create a Gemfile containing: 

source "https://rubygems.org"
source 'https://s3.amazonaws.com/bosh-jenkins-gems/'

gem "bosh_cli", "~> 1.5.0.pre"
gem "bosh_cli_plugin_micro", "~> 1.5.0.pre"

and run "bundle install" to install the bits you need.

Now, use the AWS console (or the API) to set some stuff up:

Pick a name for your MicroBOSH, e.g. "mybosh", and create a configuration file for it: 

$ mkdir -p ~/microbosh/deployments/mybosh
$ vi ~/microbosh/deployments/mybosh/micro_bosh.yml

Here's an example micro_bosh.yml file: 

name: mybosh
logging:
  level: DEBUG
network:
  type: dynamic
  vip: A.B.C.D
resources:
  persistent_disk: 4096
  cloud_properties:
    instance_type: m1.medium
cloud:
  plugin: aws
  properties:
    aws:
      access_key_id: YOURKEY
      secret_access_key: YOURSECRET
      region: ap-southeast-2
      ec2_endpoint: ec2.ap-southeast-2.amazonaws.com
      default_security_groups:
      - bosh
      default_key_name: mybosh
      ec2_private_key: /home/ubuntu/.ssh/mybosh.pem
apply_spec:
  agent:
    blobstore:
      address: A.B.C.D
    nats:
      address: A.B.C.D
  properties:
    aws_registry:
      address: A.B.C.D

You'll need to :

Now, you need a MicroBOSH image to install, so download the latest MicroBOSH stemcell: 

$ cd ~/microbosh
$ curl -O http://bosh-jenkins-artifacts.s3.amazonaws.com/micro-bosh-stemcell/aws/latest-micro-bosh-stemcell-aws.tgz

Now it's time to crank up the MicroBOSH!

Be patient; it takes a while. Actually, you should consider installing a terminal multiplexer like tmux on your inception server, and running this step within a tmux session, just in case you get disconnected while the deployment is in progress. 

$ cd ~/microbosh/deployments
$ bosh micro deployment mybosh/
Deployment set to '/home/ubuntu/microbosh/deployments/mybosh/micro_bosh.yml'

$ bosh -n micro deploy ../latest-micro-bosh-stemcell-aws.tgz

Verifying stemcell...
File exists and readable                                     OK
Using cached manifest...
Stemcell properties                                          OK

Stemcell info
-------------
Name:    micro-bosh-stemcell
Version: 776

Deploy Micro BOSH
  unpacking stemcell (00:00:16)
  uploading stemcell (00:10:48)
  creating VM from ami-c51380ff (00:00:32)
  waiting for the agent (01:01:20)
  create disk (00:04:16)
  mount disk (00:00:06)
  stopping agent services (00:00:01)
  applying micro BOSH spec (00:00:19)
  starting agent services (00:00:00)
  waiting for the director (00:00:18)
Done             11/11 01:18:06
WARNING! Your target has been changed to `https://55.251.169.14:25555'!
Deployment set to '/home/ubuntu/microbosh/deployments/mybosh/micro_bosh.yml'
Deployed `mybosh/micro_bosh.yml' to `https://mybosh:25555', took 01:18:06 to complete

Note the id of the AMI produced; you can use this for future MicroBOSH deployments in the same region, bypassing the stemcell download and conversion processes, e.g. 

$ bosh -n micro deploy ami-c51380ff

Actually, if you're really lucky, Pivotal might have already baked a MicroBOSH AMI in your target region, in which case you can just use that, and save yourself a lot of time (and network traffic). 

$ AWS_REGION=us-east-1
$ curl http://bosh-jenkins-artifacts.s3.amazonaws.com/last_successful_micro-bosh-stemcell-aws_ami_$AWS_REGION
ami-427b092b

At time of writing, they're only baking AMIs for us-east-1, so for other regions you'll have to resort to downloading the stemcell, as described above.

Once your MicroBOSH is running, you should be able to connect to it: 

$ bosh target https://55.251.169.14:25555
Target set to `mybosh'
Your username: admin
Enter password: *****
Logged in as `admin'

$ bosh status
Config
             /home/ubuntu/.bosh_config

Director
  Name       mybosh
  URL        https://55.251.169.14:25555
  Version    1.5.0.pre.776 (release:6191c586 bosh:6191c586)
  User       admin
  UUID       eac3cf02-845d-4817-aa55-7626a071304a
  CPI        aws
  dns        enabled (domain_name: microbosh)
  compiled_package_cache disabled
  snapshots  disabled

Deployment
  not set

Stay tuned for the next exciting episode!

powered by Disqus

Feedback