Build Your Own Jenkins! Introducing Custom WAR/Docker Package

I would like to introduce Custom WAR Packager - a new tool for Jenkins administrators and developers. This tool allows packaging custom Jenkins distributions as WAR files, Docker images and Jenkinsfile Runner bundles. This tool allows packaging Jenkins, plugins, and configurations in a ready-to-fly distribution. Custom WAR packager is a part of the Ephemeral Jenkins controller toolchain which we presented in our A Cloud Native Jenkins blogpost. This toolchain is already used in Jenkins X to package serverless images .

In this blog post, I will show some common use cases for Custom WAR Packager.

As with Jenkins itself, Custom WAR Packager started as a small development tool. For a long time it was a problem to run integration testing in Jenkins. We have three main frameworks for it: Jenkins Test Harness , Acceptance Test Harness and Plugin Compatibility Tester . All these frameworks require a Jenkins WAR file to be passed to them to run tests. What if you want to run Jenkins tests in a custom environment like Amazon AWS? Or what if you want to reuse existing Jenkins Pipeline tests and to run them against Pluggable Storage to ensure there are no regressions?

This is not just an idle question. There are major activities happening in the Jenkins project: Cloud Native Jenkins, Jenkins Evergreen and Jenkins X. All these activities require a lot of integration testing to enable continuous delivery flows. In order to do this in existing test frameworks, we needed to package a self-configuring WAR file so that it will be possible to run integration tests in existing frameworks. That is why Custom WAR Packager was created in April 2018. Later it received support for packaging Docker images, and in September 2018, it also got support for Jenkinsfile Runner which was created by Kohsuke Kawaguchi and then improved by Nicolas de Loof .

Custom WAR packager is a tool which is available as CLI Executable, Maven Plugin or Docker package. This tool takes input definitions and packages them as requested by the user. Everything is managed by a YAML configuration file:

The tool supports various types of inputs. The list of plugins can be passed via YAML itself, pom.xml , or a BOM file from JEP-309 . Custom WAR Packager supports not only released versions, but also builds deployed to the incremental repository (CD flow for Jenkins core and plugins - JEP-305 ) and even direct builds by Git or directory path specifications. It allows building packages from any source, without waiting for official releases. The builds are also pretty fast, because the plugin caches in the local Maven repository by using commit IDs.

Custom WAR packager also supports the following self-configuration options:

• YAML files for Jenkins Configuration as Code

• Groovy Hooks (e.g. init hooks for pre-configuration)

• System properties

WAR packaging happens by default every time the repo is built. Generally Custom WAR Packager repackages all inputs into a single WAR file by following conventions defined in the Jenkins core and the JCasC plugin.

Sample configuration:

<code data-lang="yaml">bundle:
  groupId: "<span style="color:#ff0000;">io.jenkins.tools.war-packager.demo</span> "
  artifactId: "<span style="color:#ff0000;">blogpost-demo</span> "
  vendor: "<span style="color:#ff0000;">Jenkins project</span> "
  description: "<span style="color:#ff0000;">Just a demo for the blogpost</span> "
war:
  groupId: "<span style="color:#ff0000;">org.jenkins-ci.main</span> "
  artifactId: "<span style="color:#ff0000;">jenkins-war</span> "
  source:
    version: 2.138.2
plugins:
  - groupId: "<span style="color:#ff0000;">io.jenkins</span> "
    artifactId: "<span style="color:#ff0000;">configuration-as-code</span> "
    source:
      # Common release
      version: 1.0-rc2
  - groupId: "<span style="color:#ff0000;">io.jenkins</span> "
    artifactId: "<span style="color:#ff0000;">artifact-manager-s3</span> "
    source:
      # Incrementals
      version: <span style="color:#ff0000;">1.2-rc259.c9d60bf2f88c</span> 
  - groupId: "<span style="color:#ff0000;">org.jenkins-ci.plugins.workflow</span> "
    artifactId: "<span style="color:#ff0000;">workflow-job</span> "
    source:
      # Git
      git: <span style="color:#ff0000;">https://github.com/jglick/workflow-job-plugin.git</span> 
      commit: <span style="color:#ff0000;">18d78f305a4526af9cdf3a7b68eb9caf97c7cfbc</span> 
  # etc.
systemProperties:
    jenkins.model.Jenkins.slaveAgentPort: "9000"
    jenkins.model.Jenkins.slaveAgentPortEnforce: "<span style="color:#ff0000;">true</span> "
groovyHooks:
  - <span style="color:#ff0000;">type: "init"</span> 
    id: "<span style="color:#ff0000;">initScripts</span> "
    source:
      dir: <span style="color:#ff0000;">src/main/groovy</span> 
casc:
  - <span style="color:#ff0000;">id: "jcasc"</span> 
    source:
      dir: <span style="color:#ff0000;">casc.yml</span>  

In order to do the Docker packaging, Custom WAR Packager uses the official jenkins/jenkins Docker images or other images using the same format. During the build the WAR file gets replaced by the one built by the tool. It means that ALL image features are available for such custom builds: plugins.txt , Java options, Groovy hooks, etc.

## ...
## WAR configuration from above
## ...

buildSettings:
  docker:
    build: <span style="color:#ff0000;">true</span> 
    # Base image
    base: "<span style="color:#ff0000;">jenkins</span> <span style="color:#ff0000;">/jenkins:2.138.2</span> "
    # Tag to set for the produced image
    tag: "<span style="color:#ff0000;">jenkins/custom-war-packager-casc-demo</span> "

For example, this demo shows packaging of a Docker image with External Build Logging to Elasticsearch. Although the implementations have been improved as a part of JEP-207 JEP-210 , you can check out this demo to see how the Docker image does self-configuration, connects to a Elasticsearch and then starts externally storing logs without changes in build log UIs. A Docker Compose file for running the entire cluster is included.

This is probably the most tricky mode of Jenkinsfile Runner. In March a new Jenkinsfile Runner project was announced in the developer mailing list. The main idea is to support running Jenkins Pipeline in a single-shot controller mode when the instance just executes a single run and prints outputs to the console. Jenkinsfile Runner runs as CLI or as a Docker image. Custom WAR Packager is able to produce both, though only Docker run mode is recommended. With Jenkinsfile Runner you can run Pipelines simply as…​

<code data-lang="sh">docker run --rm -v $PWD/Jenkinsfile:/workspace/Jenkinsfile acmeorg/jenkinsfile-runner 

When we started working on Ephemeral (aka "single-shot") controllers in the Cloud Native SIG, there was an idea to use Custom WAR Packager and other existing tools (Jenkinsfile Runner, Jenkins Configuration as Code, etc.) to implement it. It would be possible to just replace Jenkins core JAR and add plugins to Jenkinsfile Runner, but it is not enough. To be efficient, Jenkinsfile Runner images should start up FAST, really fast. In the build flow implementation we used some experimental options available in Jenkins and Jenkinsfile Runner, including classloader precaching, plugin unarchiving, etc. With such patches, Jenkins starts up in few seconds with configuration as code and dozens of bundled plugins.

So, how to build custom Jenkinsfile Runner images? Although there is no release so far, it is not something which stops us as you see above.

<code data-lang="yaml">##...
## WAR Configuration from above
##...

buildSettings:
  jenkinsfileRunner:
    source:
      groupId: "<span style="color:#ff0000;">io.jenkins</span> "
      artifactId: "<span style="color:#ff0000;">jenkinsfile-runner</span> "
      build:
        noCache: <span style="color:#ff0000;">true</span> 
      source:
        git: <span style="color:#ff0000;">https://github.com/jenkinsci/jenkinsfile-runner.git</span> 
        commit: <span style="color:#ff0000;">8ff9b1e9a097e629c5fbffca9a3d69750097ecc4</span> 
    docker:
      base: "<span style="color:#ff0000;">jenkins/jenkins:2.138.2</span> "
      tag: "<span style="color:#ff0000;">onenashev/cwp-jenkinsfile-runner-demo</span> "
      build: <span style="color:#ff0000;">true</span>  

You can find a Demo of Jenkinsfile Runner packaging with Custom WAR Packager here .

There are many other features which are not described in this blog post. For example, it is possible to alter Maven build settings or to add/replace libraries within the Jenkins core (e.g. Remoting). Please see the Custom WAR Packager documentation for more information. There are a number of demos available in the repository.

If you are interested to contribute to the repository, please create pull requests and cc: @oleg-nenashev and Raul Arabaolaza who is the second maintainer now working on Jenkins test automation flows.

There are still many improvements that can be made to the tool to make it more efficient:

• Add upper bounds checks for transitive plugin dependencies so that conflicts are discovered during the build

• Allow passing all kinds of system properties and Java options via configuration YAML

• Improve Jenkinsfile Runner to improve performance

• Integrate the tool into Jenkins Integration test flows (see essentialsTest in the Jenkins Pipeline library).

Many other tasks could be implemented in Custom WAR Packager, but even now it is available to all Jenkins users so that they can build their own Jenkins bundles with it.

See how Jenkins configuration as code helps Jenkins admins

Download the new whitepaper on Jenkins, Docker and DevOps

Read how Jenkins X is key to CI/CD on Kubernetes