Continuous Integration and Deployment to Amazon OpsWorks

This is a republished guest blog post by Edgars Lazdins. He is director at Novo IT – a Web Development Agency from Latvia. You can find his original article here.

Here, at Novo IT, we love using Amazon OpsWorks for deploying our internal projects. With OpsWorks, we can easily segregate our development environments in Stacks and control how each project gets built via Chef recipes. OpsWorks binds directly with your code repository of choice. When you initiate a new build, it will pull in the latest changes and build them for you.

One task, that is not immediately obvious how to solve, is triggering an OpsWorks build remotely from the command line, or from a build server. This article will explain how we do exactly this, using the excellent Codeship service.

We will make use of the AWS Identity and Access Management (IAM) feature to create a new user for our deployments. This is preferable to using, say your root account, as the new account will only have just enough permissions to carry out OpsWorks deployments.

First, we go to IAM, click on Groups and then Create New Group. This brings up the following screen:

[caption id="attachment_416" align="alignnone" width="885"]

Creating a new AWS IAM group[/caption]

This group will provide all its members with access to OpsWorks. A name similar to "opsworks-users" makes sense. Once you've picked a name, click on Continue and we are presented with a Policy selection screen. At this point, we can select from the list of available policies or create a fully custom policy. In our case, the existing "AWS OpsWorks Full Access" policy is just what we need, so let's select it:

[caption id="attachment_417" align="alignnone" width="885"]

Select the AWS OpsWorks Full Access Policy[/caption]

Now that we have an IAM group in place, let's create a user we will use for deployment. On the main IAM screen, click on Users and Create New Users, this brings up the following screen:

[caption id="attachment_418" align="alignnone" width="885"]

Create a new AWS IAM user[/caption]

Choose a sensible name and generate your new user. It is very important to download the security credentials for your new user. We will need these for deployment, namely the Access Key ID and the Secret Access Key.

Finally, add the new user to our opsworks-users group, which will give the account access to our OpsWorks deployments. This can be done by first clicking on the user, then going to the Groups tab and clicking on the Add User to Groups button.

There are some additional details we will need before we can get to deployment.

First we need to record the Stack ID for the OpsWorks Stack we will be deploying to. We need to go to our Stack and click on the Stack Settings button:

[caption id="attachment_419" align="alignnone" width="885"]

Stack Settings[/caption]

On the next screen, the Stack ID will be shown under the label OpsWorks ID, let's record it:

[caption id="attachment_420" align="alignnone" width="885"]

Stack ID[/caption]

After that we need the Application ID for the application we will be deploying. This can be found on the OpsWorks Application page:

[caption id="attachment_421" align="alignnone" width="885"]

Application ID[/caption]

Finally, we go to the OpsWorks Instance that we will be deploying to and record it's OpsWorks ID:

[caption id="attachment_422" align="alignnone" width="885"]

Instance ID[/caption]

At this point we should have our:

• AWS Access Key

• Secret Access Key

• OpsWorks Stack ID

• OpsWorks Application ID

• OpsWorks Instance ID

Before we jump into deploying from Codeship, let's try to trigger a remote OpsWorks deployment locally from our command line.

First thing we need to do is install the AWS Command Line Interface. As the page mentions, we just need to run the following:

pip install awscli

Once the command line interface is installed, we need to initialise the AWS_ACCESS_KEY_ID environment variable with our AWS Access Key, and the AWS_SECRET_ACCESS_KEY variable with our Secret Access Key. In Linux we do as follows:

export AWS_ACCESS_KEY_ID=[Our AWS Access Key]
export AWS_SECRET_ACCESS_KEY=[Our Secret Access Key]

This will make sure that once we run AWS CLI commands, the tool will be able to log us into our AWS account automatically. Finally, we can run our deployment as follows:

aws --region='us-east-1' opsworks create-deployment --stack-id='[Our Stack ID]'
--app-id='[Our Application ID]' --instance-ids='["[Our Instance ID]"]'
--command='{"Name": "deploy"}

If we go to our stack's deployment section after running this command, we should see a new build has just started.

Once we get this far, deploying from Codeship is really easy. First we need to go to the Environment section of our Codeship Project and provide our two environment variables there:

AWS_ACCESS_KEY_ID=[Our AWS Access Key]
AWS_SECRET_ACCESS_KEY=[Our Secret Access Key]

[caption id="attachment_423" align="alignnone" width="885"]

Configure your Codeship Environment variables[/caption]

After that we go to the Deployment tab, create a new script deployment method and add the following two lines to it:

pip install awscli
aws --region='us-east-1' opsworks create-deployment --stack-id='[Our Stack ID]'
--app-id='[Our Application ID]' --instance-ids='["[Our Instance ID]"]'
--command='{"Name": "deploy"}'

[caption id="attachment_424" align="alignnone" width="885"]

Add a Deployment Method[/caption]

Codeship does not have the AWS CLI installed by default, so the first line will install it on our build server (it only takes a few seconds). The second line will trigger a new deployment on our OpsWorks Stack.

We want to thank Edgars for making this article available on our blog. How do you deploy with Codeship and what are some ways you use our script deployment method? Let us know in the comments!