Continuous Integration for Mobile Apps with Jenkins: SOASTA CloudTest for iOS Apps

One of the most popular topics we've featured in recent webinars and meetups has been how to do fully automated, gesture-aware "touch testing" of mobile applications, using Jenkins running on CloudBees with SOASTA's CloudTest Platform . This really is cloud computing squared with Jenkins in the Cloud for continuous delivery and SOASTA's CloudTest providing touch testing for mobile apps as a cloud-based service. I promised a while back to do a detailed write-up on how this all works and here it is - there are also online examples and a video demo.

If you've been following my series of blogs on CI for mobile apps with Jenkins, you'll be familiar with Stockfish by now: it's a great open source chess application and I'm going to use the iOS version as a basis for this blog. The source code for the project is on GitHub and SOASTA have a very detailed tutorial that uses the same application, if you want to drill down into all the various testing scenarios that CloudTest supports. I'd like to thank Mukul Sharma over at SOASTA, who did a great job puttting the original demo together.

To keep things simple, in the online example I have two jobs: stockfishchess-ios-install-touchtest and stockfishchess-ios-run-touchtest. The first job add support for touch testing, registers the app with the SOASTA CloudTest service, builds the iOS app and deploys it to a (tethered) iPhone/iPad device; the second runs the actual test. Note that tethering is only actually required to allow the first job to run fully automated: Apple only currently allow you to push a new app to a device without a manual OK if that device is tethered. Once the app is installed on the device, tests can be run over wi-fi although I won't cover that here.

The first job consists of a standard build fromGitHub plus the Jenkins Xcode plugin for the actual build, but the Xcode build is bracketed by two SOASTA utility commands (MakeAppTouchTestable and iOSAppInstaller ) that execute as shell commands. These utilities can be downloaded from the CloudTest Welcome page (look for the Downloads area on the right-hand side) and invoked locally on the MacOS agent, as in my example config. Alternatively, you could configure Jenkins to pull these down and unzip them on-demand, using curl: this would actually be a good idea, since it minimizes the amount of configuration needed on the agents and it ensures that the build is using the latest versions of the jar files, as SOASTA do update these fairly frequently to support new CloudTest functionality

The first adds some instrumentation support to the app to make it touch testable and registers the app version with the CloudTest service:

The second (which runs after the iOS Xcode plugin builds and packages the app) pushes the newly-built app version onto the tethered iOS device. As I explained in a previous blog, we use the Jenkins ${BUILD-NUMBER} environment variable to give a unique CFBundleVersion identifier, which can be related back to the the Jenkins job and the Git history. Finally, if the build and push succeed, in my example we call the next job in the pipeline to run the actual tests:

Here's what you should see in the console output for the job (omitting the usual preamble and the xcode output): the first snippet shows the two files (Main and AppDelegate) that are modified to make the app touch testable and the registration of the app with CloudTest (I'm using the -overwriteapp flag to allow continuous builds without manual intervention):

The second snippet (below) shows the push to the iOS device:

For this example, I've used a shorter version of the "Fool's Mate" test composition described in the SOASTA CloudTest tutorial. To record and run tests, you need to use SOASTA's browser-based TouchTest Agent on the iOS device to connect to the CloudTest service. Here's a screenshot of the CloudTest console (on the left, showing the Fool's Mate composition, with a simple test for the Checkmate! dialog at the end) and the TouchTest Agent that runs on the device:

Now let's look at the second Jenkins job, that actually runs the test composition. The configuration is simple: we execute a third downloadable SOASTA command line utility (scommand ) to connect to CloudTest and then publish the results in standard JUnit test format. A nice feature provided by the SOASTA CloudTest Jenkins plugin (also downloadable from the CloudTest Welcome page) is that test failures can link straight back to the relevant place in the CloudTest web console, so that you can see exactly where the test failed. Here's the configuration and the output from running the job (remember that you need to have the TouchTest Agent on the device connected):

As well as being an incredibly useful way to do detailed, fully automated testing for mobile apps, it's also a lot of fun to watch the tests executing on the device! Here are a few screenshots of the Fool's Mate test composition plus the test results in Jenkins: of course, in real life you'd want to run whole suites of tests and include those with your unit test, code coverage and other reports, but this should give you a feel for what is possible:

To find out more about how to do cloud-based continuous deployment and testing using Jenkins and CloudTest, please join us for our upcoming webinar, Automated Testing and Continuous Deployment for Mobile Apps in the Cloud . Kohsuke Kawaguchi, Matt Solnit of SOASTA and I will demo this technology and talk about best practice for mobile app testing and CI. I hope you can join us: learn more and register here .

Mark Prichard, Senior Director of Product Management

CloudBees

Mark Prichard is Java PaaS Evangelist for CloudBees. He came to CloudBees after 13 years at BEA Systems and Oracle, where he was Product Manager for the WebLogic Platform. A graduate of St John's College, Cambridge and the Cambridge University Computer Laboratory, Mark works for CloudBees in Los Altos, CA. Follow Mark on Twitter and via his blog Clouds, Bees and Blogs.