The problem with such a setup is not the setup itself but the process to achieve it. Configuring a machine to act as slave inside your infrastructure can be tedious and time consuming. This is especially true when the same setup has to be replicated on a large pool of slaves.
Tools for configuration management or a pre-baked image can be excellent solutions to this end, while containers and virtualization are also popular tools for creating generic slave environments. Containerization has risen to prominence for this purpose, with Docker being the fastest rising in popularity and ultimately most popular among Jenkins users.
What is Docker?
Docker is an open-source project that provides a platform for building and shipping applications using containers. This platform enables developers to easily create standardized environments that ensure that a testing environment is the same as the production environment, as well as providing a lightweight solution for virtualizing applications.
Docker Swarm is a clustering tool for Docker which unites Docker pools into a single virtual host.
Scalable and resilient slaves
Docker images are an easy way to define a template for a slave machine and Docker containers are lightweight enough to perform almost as well as a “bare metal” machine, making Docker a good candidate for hosting fungible slaves.
But what happens when an organization has scaled horizontally, with many masters in their installation and each needing their own slave pool?
The open-source Docker Plugin allows masters to create and automatically tear down slaves in a Docker installation, but the configuration to connect to the Docker host will need to be re-created on every existing Jenkins master and again when a new master is onboarded.
Multiple Docker hosts may also exist, so to ensure the most efficient use of these resources, all Docker hosts in an organization should be pooled together and slave containers run in on an otherwise idle host if possible to maximize the performance of the container (similar to the logic behind the Even Scheduler plugin). This sharing between Docker hosts allows masters’ jobs to be built with minimal queue time and prevents some hosts from sitting idly while others are overloaded.
This pooling is possible with Docker Swarm, but job scheduling and Swarm configuration sharing still require special integrations with Jenkins.
Shared Docker Cloud Configuration
As a part of the next release of CloudBees Jenkins Platform, Docker Swarm may be configured as a slave pool whose configuration may be shared between all managed or “client” masters in an organization. This prevents the pain of having to configure a Swarm for each master and updating all such configurations should the installation change in any way (location, FS, max containers, etc).
Like other shareable clouds, the Docker Swarm cloud can be created as an object in CloudBees Jenkins Operations Center:
This feature is included in CloudBees Jenkins Operations Center, part of the CloudBees Jenkins Platform. Contact firstname.lastname@example.org for more information.
Other plugins complement and enhance the ways Docker can be used with Jenkins. Read more about their uses cases in these blogs: