Bringing Continuous Delivery to Cloud-Scale with Jenkins, Docker and "Tiger"

At JUC London I attended Bringing Continuous Delivery to Cloud-Scale with Jenkins, Docker and "Tiger" talk by Kohsuke Kwaguchi and Harpreet Singh.

"Continuous Delivery ", "Cloud " and "Docker " - all buzzwords in the - this talk premises to be of high interest - or just vapor-ware! - room was packed; Here are my live notes

Kohsuke and Harpreet introduced the "Tiger " project they are working on (one of them asking for more and more features, the other implementing them when he's not doing a talk at some conference - I let you guess who's who).

CloudBees is focussing on Continuous Delivery (further noted "CD" for consistency). They took Tesla car as a sample to illustrate this, as a Tesla car can receive upgrades during the night to fix a technical issue identified on running cars one day before, and let users benefit the latest fixes/features with minimal delay.

To reconcile Dev and Ops tools within a single workflow to embrace all the continuous delivery process, workflow plugin is a key component to offer better flexibility. Docker is another major brick on the lego-puzzle team have to build to address CD challenge. With lightweight isolation it offers better reproducibility. a set of Docker-related plugins have been announced at JUC DC. Combined together, they allow to package the app and resources into container, and orchestrate their usage through the CD pipeline.

• build and publish docker image (with credentials support for private repositories)

• listen to dockerhub event so jenkins do trigger a build when some image is updated, to ensure everything is always up-to-date

• workflow support to make docker images and container first class citizens in workflow DSL.

Kohsuke made a live demo of such an integration. He committed a fix to a demo project, which triggered a jenkins build to publish a fresh new Docker image. DockerHub notification then do trigger the CD workflow to upgrade the production application with this up-to-date Docker image. Docker traceability do record docker image fingerprints so we can check which Docker image was used and which jenkins build did created it.

Other demonstrated use-case is about managing build environment with Docker images. Docker plugin let you use docker containers as jenkins agents. Docker Custom build environment let you control this directly from job configuration, or as a Dockerfile committed to your SCM side-by-side with project source code.

Docker definitively is a major component in Jenkins way to address the CD challenge. CloudBees is also working on addressing large scale installations with support for Docker-shared execution within CloudBees Jenkins Operation Center. Harpreet also announced plan to deliver Kubernetes support on next release. Operation Center is evolving to embrace multi-controller installation, with "promotion" for jobs to get moved from one controller to another, cross controller triggers, and such multi-controller interactions.

CloudBees product line is evolving into CloudBees platform : Team Edition for small team, Enterprise edition for larger installations, with "packs" for specific set of additional features (Amazon support for sample), and a fresh new"Tiger" project - here we go - aka Jenkins-as-a-Service, dedicated to big companies.

DEV@Cloud already do offer such a service with thousands jenkins controllers hosted on Amazon and elastic build agent infrastructure. Tiger goal is to offer the same experience behind company firewall. Multi-tenanted controllers and agents provisioned on-demand without administration hell, is built on top of CloudBees platform so do benefit all the tooling provided by CloudBees Jenkins Enterprise (security, monitoring, visualization).

Kohsuke made a quick demo of this new product. From CloudBees Jenkins Operation Center web UI he provisioned a fresh new client controller. Tiger is managing the underlying infrastructure - based on Mesos and Docker containers - to find adequate "box" to host this new instance and storage bucket, and is sharing build resources the same way. Within the minute you get a fresh new jenkins controller setup, ready to host team jobs and builds. Tiger is moving jenkins to the Cloud-scale with such a multi-tenant distributed solution.

So, Docker again. Seems this is not about the Tiger I expected but actually some Tiger Whale...

Jenkins / Docker / Continuous Delivery story is just starting, and lot's more feature and tools integration will come to offer simpler/better/faster (Daft Punk TM ) Continuous Delivery.

Nicolas De Loof
Hacker
CloudBees