Under the Hood of Jenkins X - A CI/CD Solution for Cloud Applications on Kubernetes
First, we articulated the business value that Jenkins X in automating Continuous Everything for Kubernetes . We encourage you to read it first.
If you recall, Jenkins X as an automated CI/CD solution for cloud applications on Kubernetes. Jenkins X automates the installation, configuration, and upgrades of tools, so that users can learn at their own pace. This vastly simplifies the user experience, since users don’t have to stomach all the various pieces on Day One, and can learn at their own pace.
With the built-in automation, teams can focus on releasing quality and secure software frequently and predictably to customers, without worrying about the shipping logistics. At CloudBees, we think it will be easier for our users if we tease things apart even before you start wondering what’s under the hood. This blog provides a quick overview on some of those “undercover agents,” including Jenkins itself, so that you can seamlessly continue on your journey in the continuous paradigm!
Jenkins X introduces the jx command line - a single command to rule them all.
You can either:
Install the “jx” binary on your local machine, or
Use the Google Cloud shell . The shell comes with most of the tools that you will need like Git, gcloud, kubectl, etc.
Some examples of using jx are:
“jx create cluster gke “ creates a new Kubernetes cluster on Google Cloud
“jx create quickstart” creates a new app and imports the generated code into Git and Jenkins for CI/CD
“jx open jenkins” opens Jenkins in a browser
“jx sync” synchronises your local files to a devpod
“jx cloudbees” opens the CloudBees CD for Kubernetes app in a browser
You can type “jx help” to find the various operations you can perform in Jenkins X.
Manage Kubernetes applications in a predictable and repeatable fashion, which means, they:
Define the app
Install the app
Upgrade the app
Are versioned in source control, and hence can be audited
Are easy to create and publish, and hence reduce communication overhead
Make rollbacks to the last known good version stress-free (and drama-free)
In our modest example, Jenkins X creates the following Helm chart asset(s) in Git:
Let’s understand the basics of a chart repository first.
A chart repository is:
An HTTP server that houses an index.yaml file and optionally some packaged charts
A location where packaged charts can be stored and shared
Is an open-source Helm chart repository written in Go
Is Compatible with cloud storage backends like
Generates the repository index (index.yaml) based on packages found in storage
You are no longer required to maintain your own version of index.yaml
If you store your own version of index.yaml, it will be completely ignored
Provides a CLI for
Provides an API for:
Uploading a new chart version
Deleting a chart version
Uploading a new chart package to storage
Listing all the charts
Skaffold is a command line tool that:
Facilitates CD for Kubernetes applications
You can iterate on your application source code locally then deploy to local or remote Kubernetes clusters
Handles the workflow for building, pushing and deploying your application, and leverages that workflow in an automated pipeline to promote applications from test to staging to production.
Jenkins X creates the following build and deploy workflow yaml using Skaffold:
Jenkins is an open source automation tool that:
Is a self-contained Java-based program that can be used as a simple CI server or turned into a continuous delivery hub
Has a plugin ecosystem to support building, testing, and deploying any project
Requires Jenkinsfile that supports pipeline-as-code syntax/DSL in two formats:
Declarative syntax (recommended)
Jenkins X creates the following pipeline to promote through various stages:
Standardizes and automates the way we build, manage, and secure applications using containers
Creates true independence between applications and infrastructure
Creates a model for better collaboration and innovation between developers, QA, release, operations and security professionals
Jenkins X creates the following image for the containerized application:
Kubernetes runs production workloads at Google, which translates to running billions of containers per week
Among many other things, Kubernetes:
Automates deployment, scaling, and management of containerized apps
Groups containers that make up an application into logical units for easy management and discovery
Gives you freedom to take advantage of on-premise, hybrid, or public cloud infrastructure, letting you effortlessly migrate workloads
Restarts containers that fail
Replaces and reschedules containers when nodes die
Kills containers that don’t respond to health checks
Scales your application up and down:
Automatically based on CPU usage or
Manually with a command or a UI
Syncs files between our local file system and a Kubernetes cluster.
Transparently updates containers running on the cluster from your local checkout.
Enables developers to use their favorite IDEs to work from inside a cluster instead of from outside it. There is no reason to wait minutes to test code changes when you can see the results in seconds.
“jx sync” within Jenkins X allows us to sync files between our local lie system to the devpods on the Kubernetes cluster. This prevents typical problems like “works on my machine!” from slowing down business, even when everyone knows there are problems in production.
Prow is the system that handles GitHub events and commands for Kubernetes. It hails from the Kubernetes ecosystem and was created at Google. It is an event based solution that provides a loosely coupled architecture ideal for the cloud native world. It leverages ChatOps to interact with a build system. Prow manages its job configuration in Git. So, in the event of a wipeout, pipelines can be restored.
Prow includes a scalable and highly available webhook event handler that writes ProwJob CRDs to Kubernetes based on git events, so that other microservices can perform operations like running CI/CD pipelines. These git events can be triggered by new PRs and issues, comments, merges and pushes.
Jenkins X includes Prow out of the box when creating clusters on GKE (Google Kubernetes Engine) with Terraform using “jx create terraform” or when using the feature flag on “jx create cluster gke --prow” or “jx install --prow” commands. Prow, along with Knative build, makes serverless Jenkins possible that addresses problems originating from static Jenkins controllers.
Knative Build is a cloud native solution that extends Kubernetes with CRDs and provides ways to build applications from source. Teams can chain steps that run sequentially in the same Kubernetes pod using different containers where they can share state between each step.
Build templates are Kubernetes CRDs and are automatically upgraded with each Jenkins X release. The Prow configuration that is generated by Jenkins X when creating or importing applications refers to a build template. Knative build, along with Prow, makes serverless Jenkins possible that addresses problems originating from static Jenkins controllers.
Teams can leverage build templates to reference the type of Kubernetes pod that is created to run builds. These templates allow teams to specify which docker image to execute your build in, what environment variables exist at build time, what service accounts and secrets are in play and what volumes should be mounted.
For those of us who have tinkered with Kubernetes, we know it is difficult to setup and manage. Jenkins X alleviates that pain and leverages the managed Kubernetes solutions offered by major cloud providers, like:
Google Cloud (GKE)
Amazon Web Services (EKS)
Microsoft Azure (AKS)
Oracle Cloud (OKE)
Let’s take the example of Google cloud. The Google cloud sdk provides the command-line interface for Google Cloud Platform products and services, like:
Gcloud, which manages authentications, local configurations, developer workflow, and interactions
Kubectl, which orchestrates the deployment and management of Kubernetes container clusters on gcloud
Jenkins X automates CI/CD for cloud applications on Kubernetes. So that engineers can focus on writing and testing products that wow the world, Jenkins X abstracts away a lot of the shipping logistics that would otherwise consume your time and energy.
Jenkins X gets you started at the speed of light, however, the onus lies on you to gradually and eventually catch up to these underlying details. While we are sure you are well on your way to design and implement resilient pipelines, we wanted to make sure you have these details at your fingertips.
Pipeline is no longer a pipe dream - so, happy coding!
Tune in to the latest DevOps Radio podcast about Jenkins X
Listen to the webinar Serverless Jenkins with Jenkins X
Download the free eBook about building cloud native applications
Stay up to date
We'll never share your email address and you can opt out at any time, we promise.