Troubleshooting a Failed Build with Codeship and SauceLabs

Written by: Alfred Nutile
5 min read

In this article, I will continue where I left off with my previous post, Laravel and Behat Using Selenium and Headless Chrome, and explain what you can do when troubleshooting a failed build.

In some cases, like the example I'll use in this article, I need a bit more than headless Chrome to figure out what's wrong with my build. Fortunately, Codeship's SSH feature comes in handy for troubleshooting. So, using SSH and SauceLabs Connect, I'll demonstrate how I interacted with my running PHP server on Codeship.

In my previous article, I discussed a real example where I had everything passing locally but when I ran the build on Codeship, it failed. It was a test that required the UI since there was JavaScript involved. The issue came down to one word: true.

[http://%20asset(](http://%20asset()

On my local build, I always run my environment and my tests under https. However on Codeship I have a habit of running them under http, since I start the PHP server with this line:

php artisan serve --port=8080 -n -q &

So the addition of the word true in the asset method forces it to serve the assets under a secure URL. In Chrome, this will cause an error if the site is running in a non-secure URL.

None of my efforts before this worked: screenshots on fail, dumping the HTML with And print last response, and so on. What I really needed was to see the console tool panel that my browser offers, in this case Chrome. But I couldn't do this easily with headless Chrome.

That's when I set up SauceLabs Connect to interact with my build. I'll explain how this works step by step.

SSH into the Failing Build

On the righthand side of a build in Codeship, there's an option to SSH for debugging:

Once you log in, you just need to set things up again since it's not the same state as when your build just failed. For me, this means running these few lines to get things going:

cd clone
export SAUCELABS_NAME=YOUR_NAME_FOR_SAUCELABS
export SAUCELABS_KEY=YOUR_KEY_FOR_SAUCELABS
phpenv local 5.6
composer install
cp .env.codeship .env
php artisan serve --port=8080 -n -q &
sleep 3

Ideally this is just a script in my ci folder.

So now I have a PHP site running on port 8080 for me to interact with from SauceLabs. Regarding the SauceLabs info, you can get that from logging in to https://saucelabs.com. It will be the username you see when you click My Account...

and the KEY when you scroll down and unlock it.

We'll use this next.

Setting Up a SauceLabs Tunnel

This is a really neat feature of Linux and Unix-like systems that SauceLabs has put to good use. If you go to their Wiki, you'll find a script you can download. Typically I store this in a folder for my miscellaneous build scripts: ci. I keep a copy of this file so I can more easily just make the connection. The bash script looks like this.

#!/bin/sh
ci/bin/sc -u ${SAUCELABS_NAME} -k ${SAUCELABS_KEY} --shared-tunnel -i ci &
sleep 5

In just a moment, you'll have a tunnel to your SauceLabs account. You'll know it's connected when you log into your account and see Tunnels - 1 Active in green.

Now on the SauceLabs Dashboard, start a New Manual Test.

This will pop open a new screen. Here, I set up the remaining info, including the website's URL and port.

In a few moments, I'll get a full desktop (thanks to VNC) showing that they're using the browser.

And finally we are ready. Our server is running, the tunnel is set up, so let's interact with the site. As usual when working locally, I can click around the desktop in the browser and open up Developer Tools in Chrome and figure out what is wrong.

No more guessing, no more headless Chrome troubleshooting.

So as we experienced in my first article, Headless Chrome is great and fast, even for local work. But when things go wrong, it's best to open up a regular Chrome session and dig in as normal to see what's going on. Tools like SauceLabs and SauceLabs Connect with Codeship make this amazingly easy to do.

What About Running Tests on SauceLabs?

We've seen a great way to troubleshoot our builds. But sometimes I might want to run my build via SauceLabs instead of a local Selenium. This can also be good for troubleshooting, but it really is key for testing against the many different browsers and even mobile. If you've made it this far in this post, you're already halfway there.

For this to work, I need to have a new section in my behat.yml file. This will be a new profile called saucelabs.

saucelabs:
  extensions:
    Laracasts\Behat:
      env_path: .env.codeship
    Behat\MinkExtension:
      base_url: http://127.0.0.1:8080
      default_session: laravel
      selenium2:
        browser: chrome
        wd_host: 'ACCOUNT_NAME:ACCOUNT_KEY@ondemand.saucelabs.com/wd/hub'
        capabilities:
          platform: 'Windows 8.1'
          browser: chrome
          version: '52'

Note that you really need to save your settings above ACCOUNT_NAME:ACCOUNT_KEY. But if this is a public repo, that's too risky. You can pass it using BEHAT_PARAMS (see the following code snippet), but I find it hard to set up for multiple profiles.

export BEHAT_PARAMS="{
    \"extensions\": {
        \"Behat\\\MinkExtension\": {
            \"selenium2\": {
                \"wd_host\": \"http://${SAUCELABS_NAME}:${SAUCELABS_KEY}@localhost:4445/wd/hub\"
              }
          }
      }
  }" && vendor/bin/behat --profile=saucelabs

Moving on, I then run the SauceLabs Connect sc with some slight changes:

#!/bin/sh
ci/bin/sc -u ${SAUCELABS_NAME} -k ${SAUCELABS_KEY} --shared-tunnel &

I do not define the tunnel name -i ci so that all requests default now through the tunnel. So when I finally run behat --profile=saucelabs, the local behat will talk to the remote Selenium server and that will tunnel back to my localhost webserver via this tunnel/port forwarding that's going on.

Resources

https://wiki.saucelabs.com/display/DOCS/Test+Configuration+Options

You can view the example code here.

Stay up to date

We'll never share your email address and you can opt out at any time, we promise.