On-demand Jenkins agents with Kubernetes and the Google Container Engine

Written by: Tracy Kennedy
3 min read

In a previous series of blogs, we covered how to use Docker with Jenkins to achieve true continuous delivery and improve existing pipelines in Jenkins.

The CloudBees team and the Jenkins community have now also created the Kubernetes plugin , allowing Jenkins agents to be built as Docker images and run in Docker hosts managed by Kubernetes, either on the Google Cloud Platform or on a more local Kubernetes instance. These elastic agents are then brought online as Jenkins schedules jobs for them and destroyed after their builds are complete, ensuring controllers have steady access to clean workspaces and minimizing builds’ resource footprint.

What is Kubernetes and the Google Container Engine?

Kubernetes is an open-source project by Google which provides a platform for managing Docker containers as a cluster. Like Jenkins, Kubernetes’ orchestrating and primary node is known as the “controller”, while the node which hosts the Docker containers is called a “minion”. “Pods” host containers/services should on the minions and are defined as JSON pod files.

Source: http://blog.arungupta.me/

The Google Cloud Platform hosts the Google Container Engine , a Kubernetes-powered platform for hosting and managing Docker containers, as well as the Google Container Registry , a private Docker image registry hosted on the Google Cloud Platform. The underlying Kubernetes architecture provisions Docker containers quickly, while the Container Engine creates and manages your Kubernetes clusters.

Elastic, custom, and clean: Kubernetes agents

As the demand on a Jenkins controller increases, often so too do the build resources required. Many organizations architect for this projected growth by ensuring that their build/test environments are fungible , and therefore easily replaced and templated (e.g. as Docker images). Such fungibility makes agent resources highly scalable and resilient should some go offline or new ones need to be created quickly or automatically.

Kubernetes allows Jenkins installations to leverage any of their Docker agent images as templates for on-demand agent instances, which Jenkins can ask Kubernetes to launch as needed. The Kubernetes plugin now supports launching these agents in any Kubernetes instance, including the Google Cloud Platform’s Container Engine.

Once a Kubernetes Pod running the agent container is deployed, the Jenkins jobs requesting that specific agent via traditional labels are built inside the Pod’s agent container. Kubernetes then brings the agent’s Pod offline after its build completes.

Where do I start?

  1. The Kubernetes plugin is an open-source plugin, so it is available for download from the open-source update center or packaged as part of the CloudBees Jenkins Platform.

  2. The Google Container Engine offers a free trial .

  3. The Google Container Registry is a free service.

  4. Other plugins complement and enhance the ways Docker can be used with Jenkins. Read more about their uses cases in these blogs:

    • Docker Build and Publish Plugin

    • Docker Agents with the CloudBees Jenkins Platform

    • Jenkins Docker Workflow DSL

    • Docker Traceability

    • Docker Hub Trigger Plugin

    • Docker Custom Build Environment plugin

Tracy Kennedy
Associate Product Manager
CloudBees

Tracy Kennedy is an associate product manager for CloudBees and is based in Richmond.

Stay up to date

We'll never share your email address and you can opt out at any time, we promise.