On-demand Jenkins slaves with Kubernetes and the Google Container Engine

In a previous series of blogs, we covered how to use Docker with Jenkins to achieve true continuous delivery and improve existing pipelines in Jenkins.

The CloudBees team and the Jenkins community have now also created the Kubernetes plugin, allowing Jenkins slaves to be built as Docker images and run in Docker hosts managed by Kubernetes, either on the Google Cloud Platform or on a more local Kubernetes instance. These elastic slaves are then brought online as Jenkins schedules jobs for them and destroyed after their builds are complete, ensuring masters have steady access to clean workspaces and minimizing builds’ resource footprint.

What is Kubernetes and the Google Container Engine?

Kubernetes is an open-source project by Google which provides a platform for managing Docker containers as a cluster. Like Jenkins, Kubernetes’ orchestrating and primary node is known as the “master”, while the node which hosts the Docker containers is called a “minion”. “Pods” host containers/services should on the minions and are defined as JSON pod files.

Source: http://blog.arungupta.me/

The Google Cloud Platform hosts the Google Container Engine, a Kubernetes-powered platform for hosting and managing Docker containers, as well as the Google Container Registry, a private Docker image registry hosted on the Google Cloud Platform.  The underlying Kubernetes architecture provisions  Docker containers quickly, while the Container Engine creates and manages your Kubernetes clusters.

Elastic, custom, and clean: Kubernetes slaves

As the demand on a Jenkins master increases, often so too do the build resources required. Many organizations architect for this projected growth by ensuring that their build/test environments are fungible, and therefore easily replaced and templated (e.g. as Docker images). Such fungibility makes slave resources highly scalable and resilient should some go offline or new ones need to be created quickly or automatically.

Kubernetes allows Jenkins installations to leverage any of their Docker slave images as templates for on-demand slave instances, which Jenkins can ask Kubernetes to launch as needed. The Kubernetes plugin now supports launching these slaves in any Kubernetes instance, including the Google Cloud Platform’s Container Engine.

Once a Kubernetes Pod running the slave container is deployed, the Jenkins jobs requesting that specific slave via traditional labels are built inside the Pod’s slave container. Kubernetes then brings the slave’s Pod offline after its build completes.

Where do I start?

  1. The Kubernetes plugin is an open-source plugin, so it is available for download from the open-source update center or packaged as part of the CloudBees Jenkins Platform.
  2. The Google Container Engine offers a free trial.
  3. The Google Container Registry is a free service.
  4. Other plugins complement and enhance the ways Docker can be used with Jenkins. Read more about their uses cases in these blogs:
    • Docker Build and Publish Plugin
    • Docker Slaves with the CloudBees Jenkins Platform
    • Jenkins Docker Workflow DSL
    • Docker Traceability
    • Docker Hub Trigger Plugin
    • Docker Custom Build Environment plugin

 

Tracy Kennedy
Associate Product Manager
CloudBees 

Tracy Kennedy is an associate product manager for CloudBees and is based in Richmond.

 

 

Add new comment