Support for long-running builds in Jenkins

When we talk to prospective customers about the high-availability feature in Jenkins Enterprise, which lets a standby server take over if the running Jenkins crashes, one of the first questions to be asked is pretty simple: what happens to builds that were running during the crash? Unfortunately the answer has always been that they are aborted.

If your builds only take a few minutes, this is probably no big deal, and the Restart Aborted Builds plugin can help you keep track of which builds were interrupted this way, so that you can reschedule them. But that is cold comfort to someone running a seven-hour C++ build, or extended Selenium tests. You would really want the build to be running independently on the build machine and just have Jenkins check up on the results.

The External Monitor Job plugin was the traditional way to do this. But actually using it is pretty awkward, since Jenkins only receives the final build status. You cannot use SCM integrations like the Git plugin, or publish test results. You cannot even schedule such a build from Jenkins very easily.

To address the needs of people who have builds that are just too long to interrupt every time a Jenkins slave agent is reconnected (or Jenkins is restarted for a plugin update!), we have built an experimental plugin offering a new “long-running” job type. The configuration is almost the same as for a standard free-style job, with one difference: the part of your build that you want to run apart from Jenkins should be configured as a (Unix) shell or (Windows) batch step. Of course this script could in turn run Maven or other tools.

If the slave agent is reconnected or Jenkins restarted during this “detached build” phase, your build keeps on running uninterrupted on the slave machine (so long as that machine is not rebooted of course). When Jenkins makes contact with the slave again, it will continue to show log messages where it left off, and let the build continue. After the main phase is done, you can run the usual post-build steps, such as archiving artifacts or recording JUnit-style test results.

A beta version of this plugin has been published for Jenkins Enterprise customers. Try it out today! Enable the Jenkins Enterprise experimental update center in your global configuration screen (/configure), install the CloudBees Long-Running Build plugin, and restart Jenkins to complete the installation. Then make a new job and select Long-Running Project and note the new Detached Build section.

Because Jenkins was not originally designed for builds to survive across server or slave restarts, there are still plenty of rough edges; this release should be considered a preview to gather feedback on the usefulness of the feature. Be sure to read the release notes carefully and pay attention to the known issues. If you find bugs not already noted there, or have suggestions for improvement, just file a support ticket.

 

Jesse Glick
Developer
CloudBees

 

Comments

I am unconvinced of the 'pay for early access' model. Surely this is an example of when the early adopters should get it free and early, only when it has the "rough edges" knocked off by them should it be offered to paying customers?

Tim - this isn't paying for early access to a new feature. This is early access to people who are already paying - of a new feature.

Add new comment