Integrating Code Climate with Codeship

At Codeship, we’re pleased to be able to integrate with several third-party products across a variety of areas to ensure your CI/CD workflows are that much smoother. For example, Code Climate is an automated code coverage service, and integrating Code Climate with Codeship is fast and easy. Their documentation does a great job of providing more information, in addition to the setup instructions we have for you in this post.

Note that these instructions use the newest version of the Code Climate reporter, which is still in beta. Please view their documentation for instructions on using the older reporter. You will still need to add your API token via encrypted environment variables, as seen below, but the test configuration will work differently.

Using Code Climate with Codeship Pro

We'll cover integration with Codeship Pro and Basic here, but let's get started with Pro.

Adding Reporter ID

To begin, you'll need to add your CC_TEST_REPORTER_ID to the encrypted environment variables that you encrypt and include in your codeship-services.yml file.

Project configuration

Once your Code Climate project ID is loaded via your environment variables, install Code Climate into one of your services via your Dockerfile with the following command:

curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > "$/usr/local/bin/cc-test-reporter"
chmod +x "/usr/local/bin/cc-test-reporter"

Next, you will need to add a couple additional commands to your pipeline via your codeship-steps.yml file:

Before your test commands

- name: codeclimate_pre
  service: YOURSERVICE
  command: cc-test-reporter before-build

After your final test commands

- name: codeclimate_post
  service: YOURSERVICE
  command: cc-test-reporter after-build --exit-code $?

Parallel test coverage

Code Climate supports parallel test reports by uploading the partial result to an external service, such as S3.

To use Code Climate with parallel reporting, you'll need -- in addition to the pre-test and post-test commands above -- to add another command after your individual tests and, after all tests have completed, in your codeship-steps.yml file. Code Climate has some example scripts to walk you through this.

After each parallel test command, you’ll run a new script:

- type: parallel
  steps:
    - service: YOURSERVICE
      command: tests1.sh
    - service: YOURSERVICE
      command: tests2.sh

Note that we’re using script files to run our tests, so that we can execute the tests and export the coverage report as one command. This is because each step uses a new container, so the coverage report will not persist if the commands are separated.

Inside the new tests.sh files, you will have:

# your test commands go here
./cc-test-reporter format-coverage --output "coverage/codeclimate.$N.json"
aws s3 sync coverage/ "s3://my-bucket/coverage/$CI_COMMIT_ID"

Note that you will need to modify the S3 path (or provide an alternative storage path), as well as set the $N value value by manually declaring separate pipeline IDs.

Next, at the end of your build itself, as a new test command placed after your normal tests:

- name: codeclimate_assemble_results
  service: YOURSERVICE
  command: codeclimate-assemble.sh

Inside the codeclimate-assemble.sh file, you will have:

cc-test-reporter sum-coverage --output - --parts $PARTS coverage/codeclimate.*.json | \

Note that you will need to manually $PARTS to reflect the number of parallel threads.

Codeship Basic

Now let's take a look at how Code Climate works with Codeship Basic.

Adding Reporter ID

To begin, you'll need to add your CC_TEST_REPORTER_ID to your to your project’s environment variables. You can do this by navigating to Project Settings and then clicking on the Environment tab.

Project configuration

Once your Code Climate project ID is loaded via your environment variables, you will want to install Code Climate via your setup commands:

curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > "${HOME}/bin/cc-test-reporter"
chmod +x "${HOME}/bin/cc-test-reporter"

Next, you will need to add special Code Climate commands before and after your test commands:

Before your tests

cc-test-reporter before-build

After your final tests have run

cc-test-reporter after-build --exit-code $?

Parallel test coverage

Code Climate supports parallel test reports by uploading the partial result to an external service, such as S3.

To use Code Climate with parallel reporting, you will need -- in addition to the pre-test and post-test commands above -- to add another command at the end of your test commands, in each parallel pipeline that you run tests in, as well as a new command at the end of your build. Code Climate has some example scripts to walk you through this.

At the end of each parallel pipeline:

./cc-test-reporter format-coverage --output "coverage/codeclimate.$N.json"
aws s3 sync coverage/ "s3://my-bucket/coverage/$CI_COMMIT_ID"

Note that you will need to modify the S3 path (or provide an alternative storage path), as well as set the $N value value by manually declaring separate parallel test pipeline IDs.

At the end of your build itself, you will need to complete the parallel coverage reports in one of two ways:

• As a command, run via the custom-script deployment option. This means code coverage for parallel testing will only run on branches you have configured deployments.

• As an additional test step placed at the end of one of your parallel test pipelines. This method will require additional logic to be written to pause the script while it queries your external storage service for the existence of the appropriately named coverage reports for all the additional pipelines so that it doesn’t erroneously combine coverage reports for pipelines that are still in progress.

The code to use to end the parallel coverage report is:

 cc-test-reporter sum-coverage --output - --parts $PARTS coverage/codeclimate.*.json | \
    ./cc-test-reporter upload-coverage

Note that you will need to manually set $PARTS to reflect the number of parallel threads.

All of these commands will work best when executed from script files.