CloudBees Flow DSL: Pipeline-As-Code for Orchestrating Releases
For the last decade or so, Infrastructure-as-Code has made its way into IT, allowing you to automate configuration of cloud and infrastructure using declarative code that you can easily test, backup, version and reuse. Here are some of the values of using code to describe configurations:
Enforce versioning for better reliability, repeatability, and remediation
Make the intent or desired end state of the system more clear
Improve team efficiency by making it easier to share and collaborate
Infrastructure-as-code has been key in allowing more complex infrastructure configurations - including on-prem, virtual, and hybrid cloud. Given the benefits of infrastructure-as-code, it stands to reason that you'd want to do the same for automating pipelines and orchestrating your releases. This can include everything from using code to specify your CI pipeline, applications and environments, defining compliance and gating policies between pipeline stages, and the configuration of all the tools that are integral to your end-to-end DevOps toolchain. One of the questions I get asked the most by our customers is "How do I save my CI/CD or Release Orchestration projects in CloudBees Flow and version them? " Over the years I have written some articles about it (see Versioning Best Practices and Best Practices for Promoting to Production ), and contributed the code for the EC-Admin::projectAsCode to help with this problem. With the recent release of CloudBees Flow 6.0 , I'm thrilled to say we now support a robust DSL to bring full pipeline-as-code to CloudBees Flow !
CloudBees Flow Domain Specific Language (DSL)
CloudBees Flow provides a domain specific language (DSL), which is ideal for those times when complex processes can better be represented as code. The DSL serves as another way of interacting and working with the platform, in addition to the GUI. It allows you to define your automation processes as high-level code that is versionable, testable, and reusable. CloudBees Flow’s DSL is based on a general purpose programming language that many already know, Groovy. This gives you a lot of the flexibility due to the nature of the language itself as well as an easy access to the CloudBees Flow object library, to describe even the most complex systems. If we know anything from working with our customers, is that Automation is everywhere, and every pipeline or process is different. You want your scripting language to be flexible and extensible enough to support even your most complex scenarios, yet simple and easily understood so you can get going quickly, and easily reuse your scripts. Let me demonstrate a couple of common DSL scenarios with code samples below, so you get a sense of just how powerful this is.
The Old Way: Scripts
Let's look at how we may have tried to do something like create a project and pipeline using using code. Previously we'd use Perl (ec-perl) and we would have to write something similar to the code below:
This code works just fine. There's nothing wrong with scripting, as long as you're disciplined and well organized. But what happens if you run this code multiple times? It will fail telling you that the project already exists. So you'll have to add some clean-up code to delete your objects beforehand, or add some error management to check for the projects that exist and then modify or create a new one depending on the result. This is just a small example, but you can see how a traditional script approach requires a lot of overhead to properly maintain your code as it evolves.
The New Way: Pipeline-As-Code with CloudBees Flow DSL
Let’s now look at the same example with DSL.
To run the code above, simply save it to a file named mypipeline.groovy and type on your command line:
As you can see, in addition to being slightly shorter, the main advantage is that you can run it multiple times without fear of error. The server will automatically modify the objects that already exists. This now makes it super easy to save your CloudBees Flow objects in simple files that you can check-in to your favorite SCM. Again - this is a small example, but you can see conceptually how a declarative code configuration framework is more manageable and efficient than traditional scripts. CloudBees Flow Groovy DSL also allows:
Easy looping over a list of objects (like projects or procedures) with ".each" syntax
Transaction closures that allow you to execute a procedure created earlier
Power CloudBees Flow substitution mechanism if you run in a step
and much much more…
So does it mean that you have now to rewrite all your CloudBees Flow procedures and deploy applications in the new DSL? Fortunately not. This new version comes with an API call to generate DSL code directly from the CloudBees Flow server, very similar to the already familiar export mechanism. To accomplish this use:
Voila! This will print the DSL code in the standard output encapsulated by some XML tags, in the usual format for the CloudBees Flow API.
Using Pipeline-As-Code for Deployments
Now let's look at a slightly more complex example of modeling and deploying an application. CloudBees Flow's UI allows you to model your application, environment and deployment processes - so you can version them and reuse them over and over again, for consistent, push-button, deployments. When we use the DSL to specify our deployment model, the first part is to create a component tied to an artifact:
Then we can create the "retrieve" and "Deploy" steps inside a component process:
Then we need an application process to call the installation of our newly created component:
For simplicity and brevity, this example is incomplete. For instance, it's missing the Environments and their mapping to the app. But hopefully you get the gist.
For complete and more advanced examples, check our DSL GitHub repository , and if you need any assistance, please don't hesitate to ask us .
The full DSL documentation is available here.
If you want to learn all about the DSL - including advanced use cases and best practices - be sure to register for the upcoming DevOps Enterprise Summit . During the conference, CloudBees customers can take advantage of 2 full days of training - which cover DSL, along with a ton of other goodies to advanced your knowledge.
Stay up to date
We'll never share your email address and you can opt out at any time, we promise.