Continuous Integration for Mobile Apps with Jenkins: Promoted Builds, the QA Process and Beta Distribution

In previous blogs in this series, we’ve looked at how to use Jenkins in the cloud to achieve continuous integration for mobile app development, for both the iOS and Android platforms.  In this installment, I’d like to look at an example of ‘full lifecycle’ CI to talk about a couple of important areas that I haven’t covered yet:

  1. How to implement effective processes for QA of mobile app development using Jenkins; in this context, I want to introduce the concept of Promoted Builds and show how these can be used to provide “gating” mechanisms to ensure quality standards are met;
  2. A slightly deeper look at what’s involved in distributing applications to beta testers in the field, gathering feedback and maintaining control over the distribution; in this example, I am going to use Zubhium to push beta builds of an Android app out to selected testers.

As with the earlier blogs in this series, you can visit Cloudbees to look at the Jenkins configurations that I’ll be discussing.  For this exercise, I’m sticking with the chess scenario I used in the previous blogs, but we’ll work with a simpler, more lightweight chess app based on Peter Osterlund’s CuckooChess engine. The source code is available online as part of the DroidFish project: I have made a copy of the source code on GitHub.  For the purposes of this blog, you can ignore the two projects, CuckooChess and CuckooChessEngine: I built the cuckoochess.jar library and included it in CuckooChessAPK/libs - CuckooChessAPK is the Android project itself.

These are the three Jenkins jobs I’ll be discussing in this blog, with links to the configurations if you would like more detail:

  1. cuckoochess-android: the main developer build job
  2. cuckoochess-android-matrix: a background QA build using multiple Android SDK versions
  3. cuckoochess-android-release: pushes the built .apk archive to Zubhium for beta distribution

Let’s start with cuckoochess-android, which gets executed every time anyone pushes a change to the GitHub repository.  I’ve added a custom_rules.xml file to the normal Android build: this is a standard placeholder in build.xml that allows you to add custom Ant targets to the build.  I’ve added the following code to set the android:versionCode value in AndroidManifest.xml based on the Jenkins build number, using ${env.BUILD_NUMBER}.  It’s good practice to label your app builds in this way, so that you can easily identify the exact Jenkins build and source code version used for a particular app version: it’s especially important where you will be pushing different builds out for field beta testing and you need to check any feedback or crash data against the build system.  Here’s the code, which I adapted from this helpful AlienZombiePirate.com blog, which has more detail and additional examples:

Version code: ${app.version.code}
                     match='${matchVersionCode.start}[^"]*${match.end}' 
                     replace="${matchVersionCode.start}${app.version.code}${match.end}" />
  
With this automatically included in the Android ant build, I can just add the set-version-code ant target when building the project and that will set android:versionCode to be the Jenkins build number.  The actual Android build is simple and follows the steps I covered in this earlier blog.  Note also that as a post-build step I've told Jenkins to archive the .apk archive that will be built: we will need this to be available for subsequent build jobs to use.

This cuckoochess-android project provides a relatively quick check for the developer that the app will build, but that’s not the end of the story.  Behind the scenes Jenkins is configured to run a series of other build jobs that will do more extensive testing.  In this example, to keep things simple, I have set up a multi-configuration job that repeats the build process using all the Android SDKs back to 2.3.3 (API levels 10 through 16), to check for backwards compatibility.  In reality, I would expect a number of these jobs, involving test code, multi-device testing using emulators and/or tethered devices as well as multiple locales (as discussed in this earlier blog), but you get the idea.   This screenshot shows the multiple Android versions under test: you can see how this is configured here:

 

multiple Android versions configuration

 

This job is a downstream project triggered by a successful build of the original cuckoochess-android build, meaning that it will run automatically whenever that build completes without error.  All this can happen entirely in the background as far as the developer is concerned: these builds run in the cloud using on-demand resources, fully-automated and “lights-out”.  However, these additional builds jobs and tests must complete successfully before the original build can be used as the basis for a beta distribution and we might also want to include a manual approval step.  In Jenkins, you can do this using the Promoted Builds Plugin.  Here’s how it’s configured for the cuckoochess-android build job:

Promoted Builds Plugin

 

As you can see, there are two conditions for the build to be promoted (and the cuckoochess-android-release project to be triggered): the matrixed multi-version build job must run successfully and there needs to be a manual approval by a named approver (me, in this example).  Once both jobs have run successfully, I can select the cuckoochess-android build and then click “Promotion Status”, which will show me that the downstream build (cuckoochess-android-matrix) has passed, but that the build has not yet been promoted, as it is still awaiting manual approval:

cuckoochess-android-matrix

 

At this point, I could (and in reality, as a release manager would definitely want to) fetch the built artifact (the Android .apk archive) from the repository, install to my device or emulator and check the packaging, branding and everything else that needs my sign-off.  Then I can go ahead and approve the build by clicking on the button and the build will be promoted (it gets a gold star!):

Android build promotion history

 

This build promotion automatically triggers the final Jenkins job (cuckoochess-android-release) in this mini-process: this fetches the .apk archive from the repository (note that this is the exact binary that was tested previously) and uploads it to the Zubhium service.  Here’s the important part of the config:

Zubhium service

 

Now I can go to the Zubhium console and view the new build (in this case, CuckooChess-26 V1.05, where 26 is derived from the Jenkins build number as described above), ready to be pushed to my beta test group.  There are lots of cool things I can do with Zubhium: I can enable automatic crash reporting, collect user feedback and also provide a “kill switch” to disable beta builds once newer versions are available - and in every case, I can track any and all feedback directly to the relevant Jenkins build.

 

Zubhium console

 

If you’re working with iOS apps, rather than Android, then there are other products that also have Jenkins plug-ins and which will allow you to do similar things for iPhone/iPad: check out TestFlight and Appaloosa for starters.

Mark Prichard, Senior Director of Product Management
CloudBees

Mark Prichard is Java PaaS Evangelist for CloudBees. He came to CloudBees after 13 years at BEA Systems and Oracle, where he was Product Manager for the WebLogic Platform. A graduate of St. John’s College, Cambridge and the Cambridge University Computer Laboratory, Mark works for CloudBees in Los Altos, CA.
Follow Mark on Twitter and via his blog Clouds, Bees and Blogs.