Using ElectricCommander to Parallelize Build and Test Processes

When we talk about parallel builds at CloudBees, we're most likely to be talking about CloudBees Accelerator, which does an awesome job of speeding up builds automatically, giving 10-20x speedups on a regular basis. We sometimes forget to mention that ElectricCommander can also be used to parallelize build processes, with a very small amount of work.

Commander allows adjacent procedure steps to run in parallel. Because those steps can call subprocedures, whole procedures can be run in parallel.
This allows us to:

• Build on multiple platforms in parallel.

• Run multiple test suites in parallel. For example, run server tests in parallel with UI tests.

• Speed up static analysis like Coverity and Klocwork by spreading the work across multiple machines.

• Run independent parts of the build in parallel. For example, build the installer in parallel with running the unit tests.

We make extensive use of this feature in our own builds at CloudBees, to good effect. A full build of ElectricCommander 3.1 (9 platforms) is composed of 135 different commands (compiles, unit tests, making installers, doing installs, running system tests, etc.). Cumulatively, this adds up to 697 minutes of work. By running the build for each platform in a parallel step, we get this down to 203 minutes. By further parallelizing the work within each platform build (in particular, the tests), we get that down to 149 minutes, getting close to 5x improvement over the serial time.
Our customers report great success in speeding up their builds, in particular static analysis, with ElectricCommander. One customer took their static analysis phase from 2 hours 47 minutes down to 30 minutes. They brought their test time from 13.5 hours down to 47 minutes! Another customer improved their static analysis builds from 8 hours to 2 hours.

Running Steps in Parallel

Running commands in parallel is easy with ElectricCommander. When editing a procedure step, check the "Run in Parallel" box for two or more adjacent steps in a procedure and those steps will be executed in parallel. When all the parallel steps in that sequence have completed, the next serial step will start.
At first, this may not seem that powerful, until you consider that the steps don't have to be simple command invocations, they can be subprocedure calls. Below is a screen shot of the procedure we use to build Commander. There is one serial step, followed by ten parallel steps, one for each platform, plus one for the system tests, as you can see in the Parallel column. Last, there is one serial step to generate reports for the whole build. All but the last of these steps invoke other procedures, as shown in the Action column.

At runtime, each of these procedure calls are expanded and invoked in parallel, which looks something like this:

After configuring your steps to run in parallel, you can use Commander's Resource Pool feature to automatically distribute the workload over several machines.
It's also possible to use Commander's property system to allow steps to interact in interesting ways, for example, to have one part of the build wait until another part of the build completes. I'll show you how to do this in a future post.