Configure Jenkins to Scale iOS CI with On-demand VM Agents

Editors’s Note: This blog post was contributed by Manisha Arora, co-founder of Veertu, Inc.

Plenty has been written about continuous integration/continuous delivery (CI/CD) tools and practices for backend services, Web apps and more. It is easy to forget that a lot of code now runs on the palm of your hand and a lot of that is on iOS for Apple devices. It seems that sometimes these mobile platforms “miss out” on all these tools, but there are tools out there, such as those from Anka that help bring CI/CD, containers and all sorts of other good stuff to iOS. Read on…

There have been a lot of blogs written on how to configure Jenkins for iOS continuous integration – starting with plugin installation, using Fastlane and other tools to further streamline the process. This blog is not a comparison of CI tools or services available. It assumes that Jenkins is your CI tool of choice for iOS CI and continuous delivery. Instead, this blog is about the last step of the workflow agents on which the Jenkins iOS jobs run. In all the setups I’ve read, the last step executes on macOS hardware configured as static agents/nodes. Some of the common challenges with this static agent setup were discovered through our conversation with users.

Lack of consistent and isolated agent environments for all jobs

Almost all iOS CI build/test jobs irrespective of differences in application, install multiple dependencies (internal or from public repositories). Multiple runs of these jobs pretty much require re-install or fresh install of these dependencies on the Jenkins agent mac node. Often times, the public dependencies which are directly Git cloned from public repositories change between different runs of the same job, resulting in build/test failures. Multiple installs, uninstall of the dependencies can leave the Jenkins agent node in a polluted state with leakage between different job runs.

Difficulty with reproducible agent environments

When CI build/test fails due to environment-related issues (for example the wrong version of a specific library) the CI queue starts to build up rapidly and troubleshooting options become a great challenge. DevOps and Developers can’t spend too much time troubleshooting in agent environments as it impacts the overall available agent capacity. The focus is to quickly patch and fix the issue and get the CI queue moving again. There is no easy way to have reproducible environments to assist in these scenarios.

No flexibility to iterate changes in the environment

In a static agent node Jenkins CI setup, most often, when you have to test the iOS CI workflow on new releases of macOS and Xcode, you have to pre-assign some static node hardware capacity to run specific environments. It’s not possible to rollback or to dynamically rollout new build environment configurations.

These challenges can be addressed if it was possible to spin up Docker-like containers for macOS that can “install” an image in seconds, or update/switch between various stored images. Anka Build technology from Veertu enables this with a brand new hypervisor.framework based virtualization and container like tool ecosystem for macOS on the Mac hardware.

The way it works is first, you configure macOS Anka Build cloud. If you are going to use Mac hardware on-premise, then you can install and configure Anka Build solution on your Mac build and test hardware. If you are looking for a hosted solution, then you can use the hosted Anka Cloud from MacStadium.

After completing the setup of Anka Build Cloud on your Mac hardware, you will create your build and test image templates and store them in the Anka registry.

The next step is to configure the integration between your Jenkins and the Anka Build Cloud. We built a Jenkins plugin to enable this out-of-box. Install the Anka Build Cloud Jenkins plugin from the Jenkins plugin center. Once the plugin is installed, you can define your agent template definitions and associate them to your build/test iOS CI pipelines or jobs. Now, when your jobs execute, they will on-demand spin up required instances of specific macOS virtual machines (as defined by the label associated with the job) on the Anka Build Cloud Mac hardware cluster and these instances will be available as your Jenkins agent instances.

Below are the steps that describe this in more detail.

Step 1

Install the latest Jenkins Anka plugin from the Jenkins plugin center. This plugin code is also available on GitHub.

 

Step 2

Point the plugin to connect with the Anka Build macOS cloud. You will need an IP address (with port) of the Anka Build controller instance. Think of Anka Build controller as a central management server for your macOS Anka Build. It also exposes a rich set of REST APIs for integration with other third party CI systems.

Step 3

Define one or many VM template definitions for your iOS build/test jobs.

                                Selecting a specific image template to create agent template definition

The Template field in the plugin shows all the macOS image templates that you have created and pushed to the central Anka registry. You can pull these image templates on your machine, make changes to them (install new dependencies) and push them back to the Anka registry with a new tag/version.

All the templates stored in the Anka registry and their versions are available for use as agent templates in the Jenkins plugin. If you don’t select a version, the plugin will use the latest version available in the Anka registry.

 

Step 4

Associate the label defined in the agent template label to your jobs/pipelines.

Step 5

Now, execute one or multiple concurrent jobs. The Jenkins Anka plugin will start the required number of Jenkins agent macOS VMs of the type defined in the agent template definition. Every job executes in a consistent and completely isolated environment.

The central management server aka Anka controller manages the request queue for agent instances and provisions instances on available Mac hardware nodes in your Anka Build cluster so you can run concurrent jobs in Sierra, High Sierra and also different versions of Xcode on the same pool of hardware.

 


                                           Starting multiple iOS job requests

 

Anka Controller spins up ephemeral, isolated macOS VM instances from an image template on Anka Build mac cluster

                                                           Anka VMS associated as Jenkins agent

 

 

Once the job finishes, the agent VMs are deleted by the Anka Build Controller. The next iteration of the job spins up separate instances of VMs again from the image templates.

To learn more about Anka Build, sign up for a trial here.