Editor's Note: This guest blog post was reprinted with permission from the author.
When you use CloudBees CodeShip as your continuous integration (CI) server you can significantly increase the speed of your CI builds with parallel test pipelines. Pipelines allow you to run multiple commands at the same time, for instance, you can split the test suite across a few pipelines and complete the CI build much faster.
How to run parallel commands on CodeShip
Setup via CodeShip interface
One way is to define commands via the CodeShip interface. Once parallel test pipelines are enabled, each project can have multiple test pipelines that will run in parallel.
In order to run CI builds as fast as possible, we need to ensure the parallel commands will run a subset of the test suite in a way that all the commands complete at the same time to avoid a slow pipeline bottleneck. To split the test suite, we will use Knapsack Pro with Queue Mode which does a dynamic test suite split across pipelines for Ruby and JavaScript tests to keep running our tests in an optimal way across parallel pipelines (also known as CI nodes).
Example for test suite in RSpec for Ruby on Rails project
Configure test pipelines (1/2 used)
# first CI node running in parallel # RSpec tests in Knapsack Pro Queue Mode (dynamic test suite split) KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=0 bundle exec rake knapsack_pro:queue:rspec
Configure test pipelines (2/2 used)
# second CI node running in parallel # RSpec tests in Knapsack Pro Queue Mode (dynamic test suite split) KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=1 bundle exec rake knapsack_pro:queue:rspec
Example of test suite split for Cypress test runner in JavaScript
Configure test pipelines (1/2 used)
# first CI node running in parallel KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=0 $(npm bin)/knapsack-pro-cypress
Configure test pipelines (2/2 used)
# second CI node running in parallel KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=1 $(npm bin)/knapsack-pro-cypress
You can learn more about Cypress test runner for E2E tests in JavaScript in this article.
Setup via codeship-services.yml
If you use CodeShip Pro, then a parallel step group is defined by using the type: parallel header and then nesting all steps you want to be parallelized, as seen in this example below.
- type: parallel steps: - service: app command: KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=0 bundle exec rake knapsack_pro:queue:rspec - service: app command: KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=1 bundle exec rake knapsack_pro:queue:rspec
Here are more examples on how to configure CodeShip Pro parallelism.
How dynamic test suite split works
If you would like to better understand how the dynamic test suite split works and what problems it can solve check the video.
[youtube https://www.youtube.com/watch?v=hUEB1XDKEFY&w=560&h=315\]
Summary
Running tests in parallel is a fast way to lower the time of your CI build. To make it more efficient, we can split the test suite in an optimal way across CI nodes with Knapsack Pro to keep CI nodes auto balanced and run CI builds as fast as possible.
Additional resources
Read about what the future of feature releases hold
Register to learn how to add compliance and security to CloudBees CodeShip using Anchore
Build Docker containers for Go apps