CloudBees CI add-on for Amazon EKS blueprints

This blog is co-authored between Carlos Rodriguez Lopez and Ian Kurtz from CloudBees and Welly Siauw from Amazon Web Services.

Deploy CloudBees CI to Amazon Web Services (AWS) Elastic Kubernetes Service (EKS) clusters

If you are reading this, I’m sure you know Amazon Elastic Kubernetes Service (Amazon EKS) allows you to easily run tooling and plugins (commonly referred to as add-ons) within Kubernetes clusters that are developed by the Kubernetes open-source community. With such a large number of tooling choices and design patterns available to you, building a tailored Amazon EKS cluster that meets your application’s specific needs can take a significant amount of time. 

Here come Amazon EKS blueprints to the rescue! Think of them as a new way to effortlessly whip up Amazon EKS clusters that are preconfigured with everything your software delivery team needs. These blueprints are comprised of Infrastructure as Code (IaC) modules that set up teams with an immutable Amazon EKS cluster for CloudBees CI, with all the essential operational tools needed to begin running workloads, and ready to roll out across other accounts and regions in Amazon Web Services (AWS).

So, where does CloudBees CI come into play? CloudBees CI, a robust continuous integration (CI) tool, is built upon Jenkins, the widely-used CI/CD orchestrator. CloudBeese CI is designed specifically for large enterprises that already leverage Jenkins, but require additional functionality in governing, scaling, and maintaining how it is deployed and used by delivery teams. A notable, recent enhancement for CloudBees CI is High Availability (HA) mode, which introduces active-active HA and hibernation mode, allowing you to "turn off" idle controllers, so they do not incur costs when not in use. While there are numerous capabilities to call out, our focus here is understanding the connection between CloudBees CI and Amazon EKS blueprints.

CloudBees is thrilled to announce its integration into the expansive Amazon EKS blueprints community as an AWS partner add-on. This collaboration simplifies the adoption and exploration of CloudBees CI’s enterprise features through two key components:

• The deployment of CloudBees CI on modern platforms in AWS EKS is streamlined into a singular Terraform module, making the onboarding process seamless and efficient.

• A suite of blueprints for the CloudBees CI add-on module, designed for compatibility with Amazon EKS blueprints for Terraform that adhere to the EKS Best Practices Guides, ensuring optimized performance.

By integrating CloudBees CI with Amazon EKS blueprints, deployment speeds are significantly accelerated, granting quicker access to the extensive features of CloudBees CI on Amazon EKS. This partnership with AWS ensures that our mutual customer base remains at the forefront of innovation, equipped with the latest updates in both AWS and CloudBees solutions. The creation of the CloudBees CI Amazon EKS Terraform add-on, in alignment with AWS well-architected framework, creates a swift path for organizations to configure and manage fully operational Amazon EKS clusters that are bootstrapped with the operational software required to build, deploy, and operate their workloads.

So, whether you're a seasoned developer or just starting out, the CloudBees CI add-on for Amazon EKS blueprints now provides a sustainable deployment framework for onboarding enterprise Jenkins workloads with speed and at scale. Let’s see how this is done.

Note: CloudBees CI add-on for Amazon EKS blueprints replaces the legacy CloudBees CI AWS Quick Start as the solution for quick deploy in Amazon EKS. The legacy CloudBees CI AWS Quick Start has been deprecated.

Blueprint 01: Getting Started

Get started with the CloudBees CI on modern platforms in Amazon EKS by running this blueprint, which installs CloudBees CI on modern platforms and its prerequisites, to help you understand the minimum setup which includes:

• AWS Certificate Manager (ACM)

• Amazon EKS blueprints add-ons:

  • AWS Load Balancer Controller

  • ExternalDNS

  • Amazon Elastic Block Store (Amazon EBS) Container Storage Interface (CSI) driver, to allocate Amazon EBS volumes for hosting $JENKINS_HOME.

Blueprint 02: Deployment at scale

Once you are familiar with CloudBees CI blueprint add-on: Get started, this blueprint presents a scalable architecture and configuration by adding:

• An Amazon Elastic File System (Amazon EFS) drive that is required by CloudBees CI High Availability/Horizontal Scalability (HA/HS) controllers and is optional for non-HA/HS controllers.

• An Amazon Simple Storage Service (Amazon S3) bucket to store assets from applications like CloudBees CI, Velero, and Fluent Bit.

• Amazon EKS managed node groups for different workloads: CI applications, CI on-demand agents, CI spot agents, and Kubernetes applications.

• Amazon CloudWatch Logs to explode control plane logs and Fluent Bit logs.

• The following Amazon EKS blueprints add-ons:

• AWS EFS CSI Driver: Connects the Amazon EFS drive to the Amazon EKS cluster.

• AWS for Fluent Bit: Acts as an applications log router for log observability in CloudWatch.

• Cluster Autoscaler: Watches Amazon EKS managed node groups, to accomplish CloudBees CI auto-scaling nodes on Amazon EKS.

• Kube Prometheus Stack: Used for metrics observability.

• Metrics Server: This is a requirement for CloudBees CI HA/HS controllers for horizontal pod autoscaling.

• Velero: Backs up and restores Kubernetes resources and volume snapshots, which is only compatible with Amazon EBS.
  

• Cloudbees CI uses Configuration as Code (CasC) (refer to the casc folder) to enable exciting new features for streamlined DevOps and other enterprise features, such as CloudBees CI hibernation.

  • The operations center is using the CasC Bundle Retriever.

  • Managed controller configurations are managed from the operations center using source control management (SCM).

  • The managed controllers are using CasC bundle inheritance (refer to the parent folder). This "parent" bundle is inherited by two types of "child" controller bundles: ha and none-ha, to accommodate considerations about HA controllers.

CloudBees is thrilled to announce that this Terraform module and its companion blueprints leverage the cutting-edge CloudBees platform for the blueprint CI builds! CloudBees platform actions orchestrated by workflows allow us to perform automated testing for each new release (refer to the .cloudbees folder).

Amazon EKS blueprints and CloudBees provides streamlined deployment of robust CI solutions on AWS EKS, leveraging the power of IaC. This solution simplifies the process, adheres to best practices, and significantly accelerates deployment times, allowing developers at any level to excel in their Kubernetes projects, with the latest innovations and intelligent capabilities at their fingertips.