Update – 20160625

After assisting a reader I realized I should probably post an update to this article.  When I wrote it, Amazon AWS did not have a “Create Image” capability.  Some time ago this was introduced.  The steps below should still work, but they are cumbersome and should be replaced with the following steps:

  1. Select your Instance on the AWS Console
  2. Click the Actions button
  3. Select Image >Create Image

Yep, it’s now that easy.  You can also use Bundle AMI on a non-running instance if you wish to upload an AMI to the store.  Keep in mind that if you create an image of a running instance it will not be crash consistent and could have data issues.

Original article:

I’m doing this backwards.  I’m currently writing another article about Amazon EC2 and getting started by moving your existing hosting service to EC2.  However, I recently realized how much time and effort I had devoted to my EC2 Instance and getting things installed, configured, and running the way I want them.  Then I stumbled upon this forum article:  Degraded Amazon EC2 Instance.  So, your Amazon EC2 instance is fragile.  Just as fragile as anything you’d run at home.  More so, in some cases, because you don’t control any of the hardware.  Out of sight, is out of mind.  So, backups are not only a good idea, they are required.  But how do you perform image backups of your EC2 Instance so you can recover quickly to a working base image?

Preparing to Image

Security Keys

First, you will need your Security Credentials.  Amazon AWS (and EC2) have tons of different security credentials, certificates, keys, etc.  I had a hard time, at first, figuring out which credentials I required in order to create my Backup.  Here are the steps, in order, to get your Private Key and X.509 Certificate needed to perform an Amazon Machine Image (AMI) backup:

  1. Log into your Amazon AWS Management Console
  2. Click the Account button at the top
  3. Click the Security Credentials link
  4. Click the X.509 Certificates tab
  5. Click the Create a new Certificate link
  6. You will be presented with a popup to download BOTH your X.509 Certificate and your Private Key file.  It is CRITICAL that you download the Private Key file immediately, as Amazon DOES NOT retain, store, or allow you to download it again.  Make sure you download both files to your local computer:
  7. After downloading your keys, scroll to the bottom of the page and note your AWS Account ID:

You now have the two Security Credentials you require to build your AMI as well as  your AWS Account ID.  Throughout the rest article I will reference them in commands with the following notations:

  • Private Key File:  pk-PRIVATEKEY.pem
  • X.509 Certificate File:  cert-X509CERT.pem
  • AWS Account ID:  AWS-ACCT-ID

Setup the Server for Image Creation

There are two tasks to perform in order to prepare your server for AMI creation:

  1. Create a location on your server to build the AMI Bundle file: sudo mkdir /mnt/bundle
  2. Copy your two Security credentials to your Server and into the /mnt/bundle directory: cd /mnt/bundle sudo mv /home/[YOUR HOME]/cert-X509CERT.pem . sudo mv /home/[YOUR HOME]/pk-PRIVATEKEY.pem .

You’re now ready to build your Amazon Machine Image.

Creating your AMI

Building the Bundle

Run the following command to create your AMI Bundle:

sudo ec2-bundle-vol -k pk-PRIVATEKEY.pem -c X509CERT.pem -u AWS-KEY-ID -d /mnt/bundle/ -p BUNDLE-NAME.ami

This process will take a significant amount of time, you’ll get output like the following:

Please specify a value for arch [i386]: Copying / into the image file /mnt/bundle/BUNDLE-NAME.ami... Excluding: /sys/kernel/debug /sys/kernel/security /sys /proc /sys/fs/fuse/connections /dev/pts /dev /media /mnt /proc /sys /etc/udev/rules.d/70-persistent-net.rules /etc/udev/rules.d/z25_persistent-net.rules /mnt/bundle/BUNDLE-NAME.ami /mnt/img-mnt 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00253524 s, 414 MB/s mke2fs 1.41.9 (22-Aug-2009) Bundling image file... Splitting /mnt/bundle/BUNDLE-NAME.ami.tar.gz.enc... Created BUNDLE-NAME.ami.part.000 Created BUNDLE-NAME.ami.part.001 Created BUNDLE-NAME.ami.part.002 [...] Created BUNDLE-NAME.ami.part.283 Created BUNDLE-NAME.ami.part.284 Generating digests for each part... Digests generated. Unable to read instance meta-data for ancestor-ami-ids Unable to read instance meta-data for product-codes Creating bundle manifest... ec2-bundle-vol complete.

You now have an AMI Backup of your entire Instance. However YOU’RE NOT DONE! Currently, this AMI is sitting in your /mnt partition. This is a temporary storage area, so it can be deleted at any time. You need to upload this AMI to your S3 account for permanent storage.

Creating a S3 Bucket

In order to have a location at S3 to upload your AMI to, you first have to create a Bucket.  If you’re not sure what a Bucket is, or if some of the Amazon S3 terminology is unfamiliar to you, try reading the Amazon S3 Wikipedia article.  If you already have a Bucket ready to use, skip this section.

  1. Log into your AWS Management Console again
  2. Click the Amazon S3 tab in your Console
  3. Click the Create Bucket button
  4. Give your Bucket a name (BUCKET-NAME) and click Create

You now have a Bucket where you can store your AMI files.

More Credentials

You’re now dealing with Amazon S3, and another set of Credentials.  Here’s how to get the two Credentials you need for Amazon S3:

  1. Log into the Security Credentials area of your AWS Management Console (see beginning of article)
  2. Click the Access Keys tab and note both the Access Key ID (ACCESS-KEY-ID) and Secret Access Key (SECRET-ACCESS-KEY)

You’re now finally ready to upload the AMI to Amazon S3 for permanent storage.

Uploading the Bundle

Issue the following command to upload your AMI to Amazon S3:

sudo ec2-upload-bundle -b BUCKET-NAME -m BUNDLE-NAME.ami.manifest.xml -a ACCESS-KEY-ID -s SECRET-ACCESS-KEY

This process also takes a while, you’ll get output like the following:

Uploading bundled image parts to the S3 bucket BUCKET-NAME ... Uploaded BUNDLE-NAME.ami.part.000 Uploaded BUNDLE-NAME.ami.part.001 Uploaded BUNDLE-NAME.ami.part.002 [...] Uploaded BUNDLE-NAME.ami.part.283 Uploaded BUNDLE-NAME.ami.part.284 Uploading manifest ... Uploaded manifest. Bundle upload completed.

**CONGRATULATIONS **! You now have an AMI safely stored at Amazon S3.

But Wait There’s More

Now that your AMI is saved at Amazon S3, you should register it to your EC2 account, this allows you to easily spin up new Instances based on that AMI (more on that in a second).  To register your AMI follow these steps:

  1. Log into your AWS Management Console
  2. Click the AMIs link under Images
  3. Click the Register New AMI button
  4. Enter the AMI Manifest Path:  BUCKET-NAME/BUNDLE-NAME.ami.manifest.xml

You should now see your AMI listed.  If so, you’re done.

Testing the AMI

This step is optional and I’m not going to go into much detail about it.  You should always test your backups, because that’s the only way you’ll know if they will work properly when you really need them.  However, in order to test the AMI, you’ll have to spend some amount of money to spin up a new EC2 Instance using your saved AMI.  Fortunately, you probably only need it for about an hour, and that’s the beauty of Amazon EC2, because an hour of time is dirt cheap.

To test your AMI as a Spot Instance:

  1. Log into your AWS Management Console
  2. Click the Spot Requests link under Instances
  3. Click the Request Spot Instance button
  4. Click the My AMIs tab
  5. Click the Select button next to your saved AMI
  6. Run through all of the options to build an Instance (these should match your current Instance settings)
  7. Once created and Launched, log into it and verify your files, applications, and data

I hope you enjoyed this HOWTO, this is the most detailed explanation I’ve found on how to do this properly and what to expect.  If you liked it, please click one of the buttons below and tell someone about it.  Thank you!