Scripted Integration Tests with Jenkins and CloudBees

In the “mongochess” demo that I’ve used at a couple of recent events, I have a web app that exposes a set of REST services and I want a simple way to validate that the interface behavior hasn’t changed across builds.   There are some excellent automated testing tools available that can handle these interactions (for example, our friends at SOASTA have their Web Test Automation technology), but I needed something quite simple and I thought it would be a good example to show how easy it is to build custom integration tests into Jenkins build pipelines - and take advantage of the CloudBees PaaS to run them.

By the way, if you are interested in the application itself, the source code is on GitHub here and there is a brief overview of how it works in my blog Continuous Deployment for Mobile Apps with Jenkins

You can view the actual Jenkins configuration. For this blog, I’m only interested in the back-end pipeline, which involves building the chess server and running a short set of integration tests.  If you look at the Jenkins configuration for the mongo-chess-integration job , you’ll see that the Build phase has three parts, which each uses a specific plug-in:

  1. Copy artifacts from another project (using the Copy Artifact Plugin)

  2. Deploy to CloudBees (using the CloudBees Deployer Plugin)

  3. Execute Shell (using the XShell Plugin)

The build job is set up to run after the mongo-chess job that actually builds the application and the Copy Artifact plug-in copies the .war file generated by that project.   Here’s a screenshot showing the relevant section of the config:

 Jenkins configuration


The next step in the build uses the CloudBees Deployer Plugin to deploy the mongochess.war file to the CloudBees RUN@Cloud PaaS runtime.  Note that the application checks the system environment for a specific property (mongochess_mongoURI) that is used to specify the MongoURI for connecting to MongoDB and I have set this using the Override Environment button (click on Advanced…) to specify the name/value pair I want to use for the tests.  Here’s a screenshot:
CloudBees RUN@Cloud PaaS
Once the application has been deployed and is running on CloudBees, I use the XShell Plugin to run three separate scripts that execute some simple integration tests against the running application.  The scripts are stored in the /scripts folder in the project workspace and you can see them here.  These use curl to hit the REST services to create games, record moves and then playback the results.  In the mongochess demo, I included two short tests and one longer one that runs right through a game (Spassky v. Fischer, Game 13 of the Reykjavik World Championship Match in 1972, for chess fans!).  
Take a look at the script: there are a few points to be aware of:
  1. The shell scripts will run on a lightweight, JEOS (Just Enough Operating System) Linux so you shouldn’t rely on packages beyond the standard utilities.  That’s why I went with simple awk/sed commands, instead of using something like jsawk: of course you can use additional utilities or libraries, but you will need to make sure they are present in the workspace and the script sets the shell environment correctly.

  2. Remember that the last command in the script (here it’s diff) is what will return the result code to Jenkins and hence determine the success or failure of the Build step.  I forgot that and added a line of cleanup code (to remove temporary files): the result was that the build always succeeded, even though the diff output showed that the integration tests didn’t complete properly.  For our purposes here, no difference between actual/expected values causes diff to return zero and Jenkins treats that as success, anything else will cause a failure.

  3. I’ve been a little bit lazy, as I’ve hard-wired the URL endpoint for my service into the template file that contains the curl commands: obviously this is something I should pick up dynamically from the configuration.  I’ll come back to this in a follow-up blog, as there are several issues here that I would like to explore in more detail.

Finally, here you can see the output from a successful build, while here is a failed build. In the latter case, I set the mongchess_mongoURI environment variable to an invalid value, causing the first test script to fail because the application couldn’t connect to the back-end database.  If you look in the workspace (under ‘scripts’) you can see the generated files that actually ran (e.g. and the expected output I am testing against (e.g. tmpSpasskyFischer.out).
Mark Prichard, Senior Director of Product Management

Mark Prichard is Java PaaS Evangelist for CloudBees. He came to CloudBees after 13 years at BEA Systems and Oracle, where he was Product Manager for the WebLogic Platform. A graduate of St John’s College, Cambridge and the Cambridge University Computer Laboratory, Mark works for CloudBees in Los Altos, CA.  Follow Mark on Twitter and via his blog Clouds, Bees and Blogs.


Add new comment