Continuous Integration to Continuous Deployment with Jenkins and Deployit

On Wednesday, Andrew Phillips of XebiaLabs and I hosted a very successful webinar (Jumping from Continuous Integration to Continuous Deployment with Jenkins Enterprise and Deployit):  Andrew is VP of Product Management at XebiaLabs and on the webinar he did a great demo of how to use the Jenkins Deployit Plugin to manage complex, multi-platform deployments.  You can see the recording here, but he kindly provided this detailed write-up on the demo flow - many thanks, Andrew!

After demonstrating the Jenkins Enterprise CI setup, we discussed key preparation steps for expanding to continuous delivery (see the following blog post for more details on this topic):

  • Define initial pipeline scope and future expansion plan
  • Collect all application components (incl. configuration resources etc.)
  • Build environment-independent packages
  • Provide automated tests
  • Define automatable release triggers & checkpoints

We also introduced a set of questions that can serve as a useful “self-assessment” to give you insight into the progress of your preparations:

  1. What needs to be in our deployment packages?
  2. Can we retrieve these components automatically?
  3. Can we deploy the same components to all environments, automatically? 
  4. Can we keep environment information out of our builds?
  5. What are the stages in our pipeline? 
  6. How do they map to jobs? 
  7. Who owns/has access to these jobs?
  8. What triggers for/checkpoints between pipeline stages do I need? 
  9. Can I automate triggering/checkpoint validation?
  10. What happens when a pipeline stage fails?

We then proceeded to demonstrate a continuous deployment setup along these lines for two variants of the chess application:

  1. A MySQL version (chess: source code on GitHub) to a JBoss & MySQL setup
  2. A MongoDB version (mongochess: source code on GitHub) to a Tomcat & MongoDB setup

For the actual deployment of the application we used Deployit, XebiaLabs’ enterprise application release automation product.  In Deployit, we defined the two target environments, containing the appserver and databases as well as a remote test host for post-deployment smoke tests.

Deployment Environment for JBoss and MySQL
Deployment Environment for JBoss and MySQL

 

 

Deployment Environment for Tomcat/MongoDB
Deployment Environment for Tomcat/MongoDB
Managing Connection Properties
Example: Managing Connection Properties

We also defined environment-specific values for the application in Deployit. This allows us to customize our environment-independent deployment package that we will build in Jenkins and keep sensitive information such as credentials out of our build jobs:

Deployit environment settings

We have also defined release pipelines for our two applications to track them from Dev all the way through to Production:

Initial Deployment Pipeline
The Initial Deployment Pipeline

With our deployment targets set up, we can package our application in our build jobs and trigger deployments from Jenkins. Deployit integrates with Jenkins via the imaginatively-named Jenkins configuration
First, we configured the plugin to assemble various artifacts in the job’s workspace into a deployment package:

 

 

 

Jenkins config package

Next to build artifacts such as the WAR file and SQL scripts we can also include configuration resources such the datasource definition and post-deployment smoke test. We used placeholders for the datasource URL, username and password to ensure the deployment package is environment independent (these placeholders are automatically resolved against the environment configuration specified securely in Deployit):

jenkins config package detail
 

We also configured the Jenkins Deployit plugin to publish the package to Deployit (alternatively you could archive it and later import selected packages into Deployit as needed) and trigger a deployment to our Dev environment:

jenkins config import deploy
 

When the job is run, Deployit’s AutoFlow engine generates an appropriate deployment plan, executes the deployment and adds the deployment log to the build for debugging and audit:

 

 

Jenkins Console Output

 

We can also use Deployit to easily make arbitrary commands, such as the integration test scripts, independent of the target environment and run them agentlessly (useful for externally provided virtual machines where you cannot easily install a Jenkins agent, for instance).

This again allows us to decouple our pipeline jobs from the target environment, increasing security and reducing the maintenance required to keep our jobs in sync with our environments:

 

Jenkins Running arbitrary commands using Deployit
Running arbitrary commands using Deployit

 

 

Using Tokenized Scripts
Using Tokenized Scripts

Running both jobs results in integration-tested versions of our applications in their Dev environments:
 

The Final Deployment Pipeline
The Final Deployment Pipeline

From here, we can easily promote these versions to subsequent environments (we can also trigger such promotions from Jenkins, obviously):

Versioning the Deployment Pipeline
Versioning the Deployment Pipeline

As we get closer to Production, deployments are commonly restricted by release conditions that first need to be met. Deployit allows you to specify and track the status of these “release gates” - deployments to QA and Production are blocked in the above example since some of the defined conditions are not satisfied.

 

Andrew Phillips, VP Product Management
XebiaLabs
Andrew Phillips, VP Product Management XebiaLabs
 
Andrew is VP of Products for XebiaLabs, providers of the industry-leading release automation solution, Deployit. Andrew is a cloud, service delivery and automation expert and has been part of the shift to more automated application delivery platforms. Sitting on panels and driving blog and social media conversations, Andrew regularly contributes to key trend-defining technology discussions.

 

 
 
 
Mark Prichard, Senior Director of Product Management
CloudBees