let's build a testing framework!

What is a test? How do testing frameworks work? This post explores how different of a framework fit together and why they’re relevant. In this post we cover the following:

1. What is a the smallest unit of a “test”
2. How to write a reusable assertion following the expect(actual).toBe(expected) API
3. How to write tests such that we can see which test fails, while still running all tests, using the test(title, assertion) API.

Note This post was inspired by Kent C. Dodds’ JavaScript Testing Practices and Principles and course on Front End Masters. He also covers much of what I discuss below in his blog post, ”But really, what is a JavaScript test?” Highly encourage you to check them out as he’s fantastic!

What Is A Test?

At the most fundamental level, a test is just a piece of code that checks an assertion, i.e., what is expected, against the actual result and throws an error if they’re not the same. A good test is one that helps identify where the error occurred (and how to fix it).

So, if we have a sum function (with a bug) and we wanted to test it, we might do the following:

const sum = (a, b) => a - b // yes, there's a bug here
const diff = (a, b) => a - b

const { sum, subtract } = require("./math")

const result = sum(1, 4)
const expected = 5
if (result !== expected) {
  throw new Error(`We expected ${expected}, instead received ${result}`)
}

This is a test. Admittedly, it’s not a great test. It’s not repeatable, since we would have to repeat the whole thing for any other functionality we wanted to test. This pattern also crashes the entire test suite at the first failure (i.e., if we were testing a suite of functionality and the very first function fails, we wouldn’t know whether there were any other failures). But, it compares an assertion against a result and only fails if there’s a bug. If there were no bug, the code would run, never hit the exception and would could carry on its merry way.

We’ll tackle these problems one at a time.

Extract Logic For Cleaner Assertions

In our first test, we knew our result and expected values and then hard coded a comparison.

Many testing frameworks use an API that is (or is a variant of) expect(actual).toBe(expected). You can imagine many other methods similar to toBe, like isEqual, toHaveLength, etc.

How might we write that? Here’s one approach defining only the toBe method:

function expect(actual) {
  return {
    toBe(expected) {
      if (actual !== expected) {
        throw new Error(`We expected ${expected}, instead received ${result}`)
      }
    },
  }
}

This can now be invoked for multiple functions more easily:

const { sum, subtract } = require("./math")
function expect(actual) {
  /*...defined above */
}

const sumRes = sum(3, 7)
const sumExpected = 10
const diffRes = sum(7, 3)
const diffExpected = 4

expect(sumRes).toBe(sumExpected)
expect(diffRes).toBe(diffExpected)

We’re making progress. This is a repeatable pattern, but we haven’t solved the multiple failure case yet.

Test Wrappers

One way to allow multiple tests to fail in a test suite without crashing the whole program is to wrap assertions in a test function that is only called by the test framework. This is relevant because it means that our test names / titles can, and should, be long.¹

Let’s write the test method now. A common API for this function is test(title, assertion) where assertion is a callback function that invokes an assertion like the one written in the previous section.

function test(title, assertion) {
  try {
    assertion()
    console.log(`✔️ Test ${title} succeeded!`)
  } catch (e) {
    console.error(`✘ Failed while running ${title}`)
    console.error(e)
  }
}

Now, we can call our test functions in a way that will allow us to see when we hit a failure, but still test all of our assertions:

const { sum, subtract } = require("./math")
function expect(actual) {
  /*...defined above */
}
function test(title, assertion) {
  /* ...defined above */
}

test("sum: sum adds numbers", () => {
  const result = sum(3, 7)
  const expected = 10
  expect(result).toBe(expected)
})

test("diff: diff subtracts numbers", () => {
  const result = diff(7, 3)
  const expected = 4
  expect(result).toBe(expected)
})

Now, we can run our “test framework” from the command line and see our results:

% node tests.js

✘ Failed while running sum: sum adds numbers
Error: -4 is not equal to 10
    at Object.toBe (/Users/stephen/code/test-demo/tests.js:48:15)
    at /Users/stephen/code/test-demo/tests.js:35:18
    at test (/Users/stephen/code/test-demo/tests.js:56:5)
    at Object.<anonymous> (/Users/stephen/code/test-demo/tests.js:32:1)
    at Module._compile (internal/modules/cjs/loader.js:1015:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)
    at Module.load (internal/modules/cjs/loader.js:879:32)
    at Function.Module._load (internal/modules/cjs/loader.js:724:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47
✔️ Test diff: diff subtracts numbers succeeded!

We can see which tests passed (diff) and which failed (sum).

Wrap Up

At this point, we have a testing framework - albeit not a very robust one! The next step would be to create a test runner to run all of our test files to go with our testing framework. This is what Jest is by the way. To use Jest, all we would need to do is remove our implementations of test and expect (which Jest provides) and install Jest.

I may not have built the world’s most robust testing framework here, but what’s so interesting about this experiment is how it reveals the component pieces of a testing framework and sparks the imagination for where it might go. At a minimum, the next time I use an assertion library or testing framework, I’ll have a mental model for how it could be implemented, which I always find to be quite useful in understanding new APIs.

To recap:

• A test is code that checks an assertion and throws an error if the assertion fails. A good test is one that helps identify where the error occurred and how to fix it.
• A testing framework is a tool that allows for many tests and provides even better error messages.

Footnotes

• ¹ See Michael Lynch’s talk on “Why Good Developers Write Bad Tests” and specifically his plea to embrace long test names