Outside-In Testing for Ember Apps

While the actual tests themselves may not change, the are many different approaches to testing an Ember app. In this post, we are going to explore one such option: "Outside-in." This is where we drive the development of our application/feature starting at the highest level and let the tests guide us down all the way to lowest level.

In the case of our Ember app, the highest level is an acceptance test. An acceptance test is where we will test how a user actually interacts with our application. This means visiting pages, clicking things, and making sure that the page renders what we want it to. Testing these things will lead us down to integration tests. This is where we test components in a semi-isolated fashion. Finally, our "Outside-in" approach will take us down to the last type of Ember test: the unit test. This is where we test fully-isolated functionality.

Outside-in testing is a useful approach in that it allows the code to guide us through the testing process. We don't decide beforehand how many tests to write, or what exactly to test. The code will decide this for us.

Let's walk through a full-cycle example of this to give you a better idea. First, we'll need to have an Ember app to work with.

Our app is going be very simple. It will show a list of blog posts, each with a title, author, and short preview. This will be enough to illustrate the full outside-in approach, exercising each type of test along the way. Remember, your focus should be on the testing process, not the actual app itself. Let's create our app and run the generated tests to make sure everything is ready to go:

ember new outside-in
cd outside-in
ember test

Everything should be passing for now since we haven't actually done anything.

We start at the outside with our first acceptance test:

// tests/acceptance/posts-test.js
import { test } from 'qunit';
import moduleForAcceptance from 'outside-in/tests/helpers/module-for-acceptance';
moduleForAcceptance('Acceptance | posts');
test('displays the list of posts', function(assert) {
  visit('/posts');
  andThen(function() {
    const posts = find('.post');
    assert.equal(posts.length, 2);
  });
});

This test is pretty straightforward. Ember provides us with some useful test helpers that we can use within our acceptance tests. In this case, we're using the visit helper that will navigate to the given route and the andThen helper which will wait for all previous async helpers to finish executing. Then we just check to make sure that our blog posts are being rendered.

We can run just this test with ember test -m 'Acceptance | posts'. This test will obviously fail since we haven't actually done anything yet.

However, the failure message can help guide us to the next step: "Assertion Failed: The URL '/posts' did not match any routes in your application."

Let's go ahead and add that route with some fake data to get us started:

// app/router.js
import Ember from 'ember';
import config from './config/environment';
const Router = Ember.Router.extend({
  location: config.locationType
});
Router.map(function() {
  this.route('posts');
});
export default Router;
// app/routes/posts.js
import Ember from 'ember';
const POSTS = [{
  title: 'Sed Do Eiusmod Tempor',
  author: 'John Johnson',
  body: 'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.'
}, {
  title: 'Ut enim ad minim',
  author: 'Bob Robertson',
  body: 'Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?'
}];
export default Ember.Route.extend({
  model() {
    return POSTS;
  }
});
// app/templates/posts.hbs
{{#each model as |post|}}
  {{post-preview post=post}}
{{/each}}

Running the test again, we get "Assertion Failed: A helper named 'post-preview' could not be found." We need to create the component:

// app/components/post-preview.js
import Ember from 'ember';
export default Ember.Component.extend({
});

The test now fails because it isn't finding any elements with class "post." Now is when we drop down to a slightly lower level to test the component itself.

Our integration test checks that our component renders the post title and author as well as a truncated preview of the body:

// tests/integration/components/post-preview-test.js
import { moduleForComponent, test } from 'ember-qunit';
import hbs from 'htmlbars-inline-precompile';
const POST = {
  title: 'Sed Do Eiusmod Tempor',
  author: 'John Johnson',
  body: 'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.'
};
const PREVIEW = 'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor ...';
moduleForComponent('post-preview', 'Integration | Component | post preview', {
  integration: true
});
test('it has the correct class', function(assert) {
  this.set('post', POST);
  this.render(hbs`{{post-preview post=post}}`);
  assert.ok(this.$('.post').length);
});
test('it renders the post attributes', function(assert) {
  this.set('post', POST);
  this.render(hbs`{{post-preview post=post}}`);
  assert.equal(this.$('.title').text().trim(), 'Sed Do Eiusmod Tempor');
  assert.equal(this.$('.author').text().trim(), 'John Johnson');
  assert.equal(this.$('.preview').text().trim(), PREVIEW);
});

We can run just these tests with ember test -m 'Integration | Component | post preview'. Both tests will fail because we haven't actually implemented the component at all yet. We can get the first test to pass by simply adding classNames:

// app/components/post-preview.js
import Ember from 'ember';
export default Ember.Component.extend({
  classNames: ['post']
});</p>

The second test requires a little more work. Let's add a basic template:

// app/templates/components/post-preview.hbs
<h2 class="title">{{post.title}}</h2>
<div class="author">{{post.author}}</div>
<p class="preview">{{post.body}}</p>

This gets us most of the way there. However, instead of rendering a truncated preview of the post, our component is currently rendering the entire post body. Let's make use of a helper to truncate the body:

// app/templates/components/post-preview.hbs
<h2 class="title">{{post.title}}</h2>
<div class="author">{{post.author}}</div>
<p class="preview">{{truncate post.body length=80}}</p>

The tests now fail with "Assertion Failed: A helper named 'truncate' could not be found." Let's go ahead and create the helper:

// app/helpers/truncate.js
import Ember from 'ember';
export function truncate(params/*, hash*/) {
  return params;
}
export default Ember.Helper.helper(truncate);

Running the integration test once more, we see that the helper is now being used but the test is still failing because it isn't actually truncating anything. This leads us to drop down one more level where we can unit test the truncate helper's behavior in isolation.

We want our helper to do one simple thing. We want it to truncate the given text and add an ellipsis if the length of the text is greater than it should be. We'll need a few tests to make sure this working as expected:

// tests/unit/helpers/truncate-test.js
import { truncate } from '../../../helpers/truncate';
import { module, test } from 'qunit';
module('Unit | Helper | truncate');
test('it does nothing if text is not longer than the given length', function(assert) {
  const result = truncate(['short'], { length: 5 });
  const expected = 'short';
  assert.equal(result, expected);
});
test('it truncates with an ellipsis if text is longer than the given length', function(assert) {
  const result = truncate(['longer'], { length: 5 });
  const expected = 'longe...';
  assert.equal(result, expected);
});

The first test will pass as our helper is currently just returning whatever is passed in. The second one will fail. Let's get it to pass by truncating to the specified length:

// app/helpers/truncate.js
import Ember from 'ember';
export function truncate([text], { length }) {
  if (text && text.length > length) {
    return `${text.substring(0, length)}...`;
  } else {
    return text;
  }
}
export default Ember.Helper.helper(truncate);

Great! That worked. Our helper is good to go.

Our tests guided us down to the lowest level where we unit tested our helper until it did what we want. Now we can step back up a level to verify that our integration test for the component that uses that helper is also passing:

ember test -m 'Integration | Component | post preview'

Indeed, it is. Let's step back up to the highest level now and make sure our acceptance test is passing:

ember test -m 'Acceptance | posts'

We are all green! We can run the whole test suite ember test to verify that everything is working.

We've now gone full circle. We started with a high-level acceptance test and let that guide us down to a lower-level integration/component test, which then drove us even lower to a helper unit test. Once that unit test was passing, we are able to step back up a level and see that our integration test was passing. Stepping up another level, our acceptance test was passing as well.

This "outside-in" process of testing feels very natural once you get the hang of it. We don't need to think too hard about what the next step is. The test failures guide us there. The example application used here was overly simple, but the process scales up to much more complicated features. Paired with Ember's wonderful testing tools, this makes for an enjoyable way to develop your app.