We recently released version 1.3 of Declarative Pipeline , which includes a couple of significant new features such as support for sequential stages. This feature allows you to add multiple "sequential" stages in the same parallel branch. As a user, you would benefit from increased control of pipelines and be able to utilize declarative syntax for more use cases.
Sequential stages
In Declarative 1.2, we added the ability to define stages to run in parallel as part of the Declarative syntax. Now in Declarative 1.3, we’ve added another way to specify stages nested within other stages, which we’re calling "sequential stages".
Running multiple stages in a parallel branch
One common use case is running build and tests on multiple platforms. You could already do that with parallel stages, but now you can run multiple stages in each parallel branch giving you more visibility into the progress of your Pipeline without having to check the logs to see exactly which step is currently running where, etc.
You can also use stage directives, including post, when, agent and all the others covered in the Pipeline Syntax reference in your sequential stages, letting you control behavior for different parts of each parallel branch.
In the example below, we are running builds on both Windows and Linux, but only want to deploy if we’re on the controller branch.
pipeline {
agent none
stages {
stage("build and deploy on Windows and Linux") {
parallel {
stage("windows") {
agent {
label "windows"
}
stages {
stage("build") {
steps {
bat "run-build.bat"
}
}
stage("deploy") {
when {
branch "controller"
}
steps {
bat "run-deploy.bat"
}
}
}
}
stage("linux") {
agent {
label "linux"
}
stages {
stage("build") {
steps {
sh "./run-build.sh"
}
}
stage("deploy") {
when {
branch "controller"
}
steps {
sh "./run-deploy.sh"
}
}
}
}
}
}
}
}
Running multiple stages with the same agent or environment or options
While the sequential stages feature was originally driven by users wanting to have multiple stages in parallel branches, we’ve found that being able to group multiple stages together with the same agent, environment, when, etc. has a lot of other uses. For example, if you are using multiple agents in your Pipeline, but would like to be sure that stages using the same agent use the same workspace, you can use a parent stage with an agent directive on it, and then all the stages inside its stages directive will run on the same executor, in the same workspace. Another example is that until now, you could only set a timeout for the entire Pipeline or an individual stage. But by using a parent stage with nested stages, you can define a timeout in the parent’s options directive, and that timeout will be applied for the execution of the parent, including its nested stages. You may also want to conditionally control the execution of multiple stages. For example, your deployment process may be spread across multiple stages, and you don’t want to run any of those stages unless you’re on a certain branch or some other criteria is satisfied. Now you can group all those related stages together in aparent stage, within its stages directive, and have a single when condition on that parent, rather than having to copy an identical when condition to each of the relevant stages.
One of my favorite use cases is shown in the example below. In Declarative 1.2.6, we added the input directive for stages. This will pause the execution of the Pipeline until a user confirms that the Pipeline should continue, using the Scripted Pipeline input step. The input directive is evaluated before the stage enters its agent, if it has one specified, and before the stage’swhen condition, if specified, is evaluated. But if you’re using a top-level agent for most of your stages, you’re still going to be using that agent’s executor while waiting for input, which can be a waste of resources. With sequential stages, you can instead use agent none at the top-level of the Pipeline, and group the stages using a common agent and running before the stage with the input directive together under a parent stage with the required agent specified. Then, when your Pipeline reaches the stage with input , it will no longer be using an agent’s executor.
pipeline {
agent none
stages {
stage("build and test the project") {
agent {
docker "our-build-tools-image"
}
stages {
stage("build") {
steps {
sh "./build.sh"
}
}
stage("test") {
steps {
sh "./test.sh"
}
}
}
post {
success {
stash name: "artifacts", includes: "artifacts/**/*"
}
}
}
stage("deploy the artifacts if a user confirms") {
input {
message "Should we deploy the project?"
}
agent {
docker "our-deploy-tools-image"
}
steps {
sh "./deploy.sh"
}
}
}
}
These are just a few examples of the power of the new sequential stages feature in Declarative 1.3. This new feature adds another set of significant use cases that can be handled smoothly using Declarative Pipeline. In my next post, I’ll show another highly requested feature - the new ability to restart a pipeline run from any stage in that pipeline.
Andrew Bayer
CloudBees
Andrew is a long-time Jenkins contributor and plugin developer. He is an engineer at CloudBees working on various Jenkins stuff, and adding value to Jenkins every day! You can interact with him on Twitter - his favorite social media platform (because he can be a curmudgeon and no one calls him out on it). He is also on GitHub .