Build Your Own Jenkins! Introducing Custom WAR/Docker Package

I would like to introduce Custom WAR Packager- a new tool for Jenkins administrators and developers. This tool allows packaging custom Jenkins distributions as WAR files, Docker images and Jenkinsfile Runner bundles. This tool allows packaging Jenkins, plugins, and configurations in a ready-to-fly distribution. Custom WAR packager is a part of the Ephemeral Jenkins master toolchain which we presented in our A Cloud Native Jenkins blogpost. This toolchain is already used in Jenkins X to package serverless images.

In this blog post, I will show some common use cases for Custom WAR Packager.

History

As with Jenkins itself, Custom WAR Packager started as a small development tool. For a long time it was a problem to run integration testing in Jenkins. We have three main frameworks for it: Jenkins Test HarnessAcceptance Test Harness and Plugin Compatibility Tester. All these frameworks require a Jenkins WAR file to be passed to them to run tests. What if you want to run Jenkins tests in a custom environment like Amazon AWS? Or what if you want to reuse existing Jenkins Pipeline tests and to run them against Pluggable Storage to ensure there are no regressions?

This is not just an idle question. There are major activities happening in the Jenkins project: Cloud Native Jenkins, Jenkins Evergreen and Jenkins X. All these activities require a lot of integration testing to enable continuous delivery flows. In order to do this in existing test frameworks, we needed to package a self-configuring WAR file so that it will be possible to run integration tests in existing frameworks. That is why Custom WAR Packager was created in April 2018. Later it received support for packaging Docker images, and in September 2018, it also got support for Jenkinsfile Runner which was created by Kohsuke Kawaguchi and then improved by Nicolas de Loof.

What’s inside?

Custom WAR packager is a tool which is available as CLI Executable, Maven Plugin or Docker package. This tool takes input definitions and packages them as requested by the user. Everything is managed by a YAML configuration file:

The tool supports various types of inputs. The list of plugins can be passed via YAML itself, pom.xml, or a BOM file from JEP-309. Custom WAR Packager supports not only released versions, but also builds deployed to the incremental repository (CD flow for Jenkins core and plugins - JEP-305) and even direct builds by Git or directory path specifications. It allows building packages from any source, without waiting for official releases. The builds are also pretty fast, because the plugin caches in the local Maven repository by using commit IDs.

Custom WAR packager also supports the following self-configuration options:

WAR packaging

WAR packaging happens by default every time the repo is built. Generally Custom WAR Packager repackages all inputs into a single WAR file by following conventions defined in the Jenkins core and the JCasC plugin.

Sample configuration:

bundle:
  groupId: "io.jenkins.tools.war-packager.demo"
  artifactId: "blogpost-demo"
  vendor: "Jenkins project"
  description: "Just a demo for the blogpost"
war:
  groupId: "org.jenkins-ci.main"
  artifactId: "jenkins-war"
  source:
    version: 2.138.2
plugins:
  - groupId: "io.jenkins"
    artifactId: "configuration-as-code"
    source:
      # Common release
      version: 1.0-rc2
  - groupId: "io.jenkins"
    artifactId: "artifact-manager-s3"
    source:
      # Incrementals
      version: 1.2-rc259.c9d60bf2f88c
  - groupId: "org.jenkins-ci.plugins.workflow"
    artifactId: "workflow-job"
    source:
      # Git
      git: https://github.com/jglick/workflow-job-plugin.git
      commit: 18d78f305a4526af9cdf3a7b68eb9caf97c7cfbc
  # etc.
systemProperties:
    jenkins.model.Jenkins.slaveAgentPort: "9000"
    jenkins.model.Jenkins.slaveAgentPortEnforce: "true"
groovyHooks:
  - type: "init"
    id: "initScripts"
    source:
      dir: src/main/groovy
casc:
  - id: "jcasc"
    source:
      dir: casc.yml

Docker packaging

In order to do the Docker packaging, Custom WAR Packager uses the official jenkins/jenkins Docker images or other images using the same format. During the build the WAR file  gets replaced by the one built by the tool. It means that ALL image features are available for such custom builds: plugins.txt, Java options, Groovy hooks, etc.

## ...
## WAR configuration from above
## ...

buildSettings:
  docker:
    build: true
    # Base image
    base: "jenkins/jenkins:2.138.2"
    # Tag to set for the produced image
    tag: "jenkins/custom-war-packager-casc-demo"

For example, this demo shows packaging of a Docker image with External Build Logging to Elasticsearch. Although the implementations have been improved as a part of JEP-207JEP-210, you can check out this demo to see how the Docker image does self-configuration, connects to a Elasticsearch and then starts externally storing logs without changes in build log UIs. A Docker Compose file for running the entire cluster is included.

Jenkinsfile Runner packaging

This is probably the most tricky mode of Jenkinsfile Runner. In March a new Jenkinsfile Runner project was announced in the developer mailing list. The main idea is to support running Jenkins Pipeline in a single-shot master mode when the instance just executes a single run and prints outputs to the console. Jenkinsfile Runner runs as  CLI or as a Docker image. Custom WAR Packager is able to produce both, though only Docker run mode is recommended. With Jenkinsfile Runner you can run Pipelines simply as…​

docker run --rm -v $PWD/Jenkinsfile:/workspace/Jenkinsfile acmeorg/jenkinsfile-runner

When we started working on Ephemeral (aka “single-shot”) masters in the Cloud Native SIG, there was an idea to use Custom WAR Packager and other existing tools (Jenkinsfile Runner, Jenkins Configuration as Code, etc.) to implement it. It would be possible to just replace Jenkins core JAR and add plugins to Jenkinsfile Runner, but it is not enough. To be efficient, Jenkinsfile Runner images should start up FAST, really fast. In the build flow implementation we used some experimental options available in Jenkins and Jenkinsfile Runner, including classloader precaching, plugin unarchiving, etc. With such patches, Jenkins starts up in few seconds with configuration as code and dozens of bundled plugins.

So, how to build custom Jenkinsfile Runner images? Although there is no release so far, it is not something which stops us as you see above.

##...
## WAR Configuration from above
##...

buildSettings:
  jenkinsfileRunner:
    source:
      groupId: "io.jenkins"
      artifactId: "jenkinsfile-runner"
      build:
        noCache: true
      source:
        git: https://github.com/jenkinsci/jenkinsfile-runner.git
        commit: 8ff9b1e9a097e629c5fbffca9a3d69750097ecc4
    docker:
      base: "jenkins/jenkins:2.138.2"
      tag: "onenashev/cwp-jenkinsfile-runner-demo"
      build: true

 

You can find a Demo of Jenkinsfile Runner packaging with Custom WAR Packager here.

More information

There are many other features which are not described in this blog post. For example, it is possible to alter Maven build settings or to add/replace libraries within the Jenkins core (e.g. Remoting). Please see the Custom WAR Packager documentation for more information. There are a number of demos available in the repository.

If you are interested to contribute to the repository, please create pull requests and cc: @oleg-nenashev and Raul Arabaolaza who is the second maintainer now working on Jenkins test automation flows.

What’s next?

There are still many improvements that can be made to the tool to make it more efficient:

  • Add upper bounds checks for transitive plugin dependencies so that conflicts are discovered during the build

  • Allow passing all kinds of system properties and Java options via configuration YAML

  • Improve Jenkinsfile Runner to improve performance

  • Integrate the tool into Jenkins Integration test flows (see essentialsTest in the Jenkins Pipeline library).

Many other tasks could be implemented in Custom WAR Packager, but even now it is available to all Jenkins users so that they can build their own Jenkins bundles with it.

Additional resources

See how Jenkins configuration as code helps Jenkins admins

Download the new whitepaper on Jenkins, Docker and DevOps

Read how Jenkins X is key to CI/CD on Kubernetes