Literate Builds Make Configuring Jenkins Easier

Every software project has two kinds of information:

• Information that is of a wider scope than the project's source code repository;

• Information that is implicitly tied to the project's source code.

This split is important, because if you store one kind of information in the other place you open yourself up to a world of pain. Let's look at an example: “When the FooBar project is built, we need to deploy it to a staging environment.” This piece of information is wider in scope than just the FooBar project's source code. The source code does not know the IP address of the staging environment. The source code does not know the credentials used to deploy to the staging environment. Storing those things in source control is tempting… of course it is, we've all done it at some time after all. Ultimately, though, it is the wrong thing to do: you can't redeploy old revisions to verify bugs because the IP address or credentials have changed, etc.

Another example: “We build the FooBar project by running the SuperBuzzDing build tool with the Bargle command line parameter." This piece of information is very tightly coupled to the source code you are building. It may be that when you get to version 2.0 of FooBar you need to add some more bits to the command line. Or maybe the build process gets completely refactored to a different build tool entirely.

Traditionally, Jenkins has taken the approach that it does not require anything in the source code repository that you are building. A sort of loose coupling if you like. This is good up to a point. It allows you to keep the information that is greater than a single project outside of that project. Changing the staging environment does not necessitate source code changes. But that flexibility comes at a cost. We have no standard location in which to store the information that is tightly coupled to the source code. Literate builds provide such a location, and in so doing enable some very useful features.

Literate builds use the README.md file in the root of a software project to store the build instructions. In most normal projects the README file is either missing or hopelessly out of date. By using this file as the source of build instructions, we ensure that the file is kept correct and up to date. If the build tool changes from a batch file / shell script to a Makefile to an Ant build.xml to a Maven pom.xml, it doesn't matter, because the README.md will be updated with the code (or Jenkins will report a failed build).

Another important piece of information that is very relevant to the project's source code is the exact toolchain that you need to build the source code. Version 1.0 of the project may need Ruby 1.8 on Windows, version 2.0 may need Ruby 1.9 on Windows or Linux and version 3.0 may be built on Ruby 2.0 but only on Linux. This is the kind of information that people typically keep in their README files anyway. By leveraging this information, Jenkins is able to validate the build instructions against all the target environments and ensure that not only are the instructions correct, but that they are correct on all the supported platforms.

The final piece of the puzzle that literate builds provide is that you can now let Jenkins auto-create jobs for all the branches of your project. Because each branch now stores its own relevant build instructions, you can have the branch project jobs be created (and destroyed if you choose) as each branch gets created. This enables a much more flexible style of development, where creating a branch is significantly less of an administrative overhead.

To paraphrase a few Jenkins administrators: “I hate branches because I have to go and create five or six jobs for each branch, and then two weeks later the branch is gone again.” With the literate builds plugin, Jenkins will automatically create these jobs and remove them again when they are gone. The Jenkins administrator is free to go back to his or her primary job.

I will finish off this post with an example of what a literate build looks like:

<span style="white-space: pre-wrap; line-height: 1.6;">FooBar project
</span> <span style="line-height: 1.6; white-space: pre-wrap;">==============
</span> <span style="white-space: pre-wrap; line-height: 1.6;">This is the </span> FooBar<span style="white-space: pre-wrap; line-height: 1.6;"> project that allows you to block </span> Foo<span style="white-space: pre-wrap; line-height: 1.6;"> from coming into your computer.
</span> <span style="line-height: 1.6; white-space: pre-wrap;">Environments
</span> <span style="line-height: 1.6; white-space: pre-wrap;">------------
</span> <span style="white-space: pre-wrap; line-height: 1.6;">The project can be built on the following environments:
</span> <span style="white-space: pre-wrap; line-height: 1.6;">* `ruby-2.0`, `java-1.7`, `maven-3.0.5`, `rake-10.0.3`
</span> <span style="white-space: pre-wrap; line-height: 1.6;"> * `</span> linux<span style="white-space: pre-wrap; line-height: 1.6;">`
</span> <span style="white-space: pre-wrap; line-height: 1.6;"> * `windows`
</span> <span style="white-space: pre-wrap; line-height: 1.6;"> * `</span> osx<span style="white-space: pre-wrap; line-height: 1.6;">`
</span> <span style="white-space: pre-wrap; line-height: 1.6;">How to build
</span> <span style="white-space: pre-wrap; line-height: 1.6;">------------
</span> <span style="white-space: pre-wrap; line-height: 1.6;">* On `windows`
</span> <span style="white-space: pre-wrap; line-height: 1.6;"> call mvn.bat clean verify
</span> <span style="white-space: pre-wrap; line-height: 1.6;"> call rake.bat
</span> <span style="white-space: pre-wrap; line-height: 1.6;">* On `</span> linux<span style="white-space: pre-wrap; line-height: 1.6;">`</span> <span style="white-space: pre-wrap; line-height: 1.6;"> 
</span> <span style="white-space: pre-wrap; line-height: 1.6;"> </span> mvn<span style="white-space: pre-wrap; line-height: 1.6;"> clean verify; rake
</span> <span style="white-space: pre-wrap; line-height: 1.6;">* On `</span> osx<span style="white-space: pre-wrap; line-height: 1.6;">`
</span> <span style="white-space: pre-wrap; line-height: 1.6;"> </span> mvn<span style="white-space: pre-wrap; line-height: 1.6;"> clean verify; rake</span> 

The above example is both a nice, clean README.md file and a description of the build for Jenkins to use.

For more information about literate builds see the following resources:

• The Jenkins wiki page: https://wiki.jenkins-ci.org/display/JENKINS/Literate+Plugin

• The Plugin's README.md (dogfooding): https://github.com/jenkinsci/literate-plugin

• Our own wiki page

• Examples of Literate projects: https://github.com/jenkinsci/literate-api/tree/controller/src/test/resources/org/cloudbees/literate/api/v1/MarkdownModelTest

—Stephen Connolly
CloudBees
​cloudbees.com

Stephen Connolly has over 20 years experience in software development. He is involved in a number of open source projects, including Jenkins . Stephen was one of the first non-Sun committers to the Jenkins project and developed the weather icons. Stephen lives in Dublin, Ireland - where the weather icons are particularly useful. Follow Stephen on Twitter and on his blog .