jest: mocking node's fs

I hit some road blocks with nom recently which are motivating a refactoring with an eye toward reliability through testing.¹ Since nom’s primary features automate rote activities by interacting with the file system on the users’ behalf, it became clear that I would need to not only understanding testing a lot better, but specifically testing the file system. This post is an introduction to one way of doing just that using a collection of tools within Jest’s testing framework.

I started by investigating common solutions for testing the file system within the Node ecosystem. One popular approach is to use mock-fs. mock-fs uses an in-memory file system to back the fs API. This seems like a promising approach, and, it turns out is similar to the approach Vadim Dalecky, who goes by the handle streamich², has taken with memfs. He also has unionfs, spyfs, and linkfs. Small libraries that are intended to be used together for testing purposes. Based on my experience with Vadim’s libraries previously, I decided to give memfs and unionfs a shot.

Let’s start with a “simple” example: create a mock file system and then test reading a file from it.³

import { Volume } from "memfs"

// jest will mock any import of the fs module
jest.mock("fs", () => {
  const fs = jest.requireActual("fs")
  const unionfs = require("unionfs").default
  return unionfs.use(fs)
})

beforeEach(() => {
  jest.resetAllMocks()
})

test("calling fs returns a mocked call", async () => {
  //arrange
  const vol = Volume.fromJSON({ "/foo": "bar" })
  const fsMocked = require("fs")
  const fs = fsMocked.use(vol) // tell the mocked fs (via Jest) to use vol as its file system.
  // act
  const val = fs.readFileSync("/foo", { encoding: "utf8" })
  // assert
  expect(val).toBe("bar")
})

I’m following the Arrange, Act, Assert framework to writing tests (and being annoyingly explicit about it for the time being).

In the “arrange” section, I’m setting up fake file system. In this case, I’m creating a volume with a single file in the root called foo. It’s contents in total are bar.

The “tricky” part was understanding how to use the “mocked” fs module. Before even getting to this code, I used a used the module factory parameter to mock fs:

jest.mock("fs", () => {
  const fs = jest.requireActual("fs")
  const unionfs = require("unionfs").default
  return unionfs.use(fs)
})

In practice, this means that if my function were to ever use fs, instead of the Node module fs, it would be mocked with this version. Notably, I start with having jest base the mock on the actual fs module (more on this in a bit). What I actually return, however, is unionfs, which extends the API to allow for appending, or unioning (hence the name) more file systems.

Jumping down to the body of the test, in the arrange section, appending new file systems is exactly what I’m doing:

test("calling fs returns a mocked call", async () => {
    //...
    const vol = Volume.fromJSON({ "/foo": "bar" })
    const fsMocked = require("fs")
    const fs = fsMocked.use(vol) // tell the mocked fs (via Jest) to use vol as its file system.
    //...
}

fsMocked imports the module fs (which we previously told Jest to mock). Then, because it is mocked with unionfs, we tell unionfs to “use” our fake volume, vol. The result is a file system that includes a file foo in the root with the contents bar. This is why the assertion succeeds when we’ve read the file in the act section:

test("calling fs returns a mocked call", async () => {
    //...
    //act
    const val = fs.readFileSync("/foo", { encoding: "utf8" })
    // assert
    expect(val).toBe("bar")
}

Interestingly, unlike mock-fs (if I understand that API correctly), with unionfs, this setup means that I have access to the real file system as well.

While it seems reasonable that in tests we should operate with as few assumptions as possible (including which files can be found on the local disk as they can change from machine to machine), with this setup I actually can test accessing local files that really do exist.

To demonstrate this point, I added an additional assertion to my test:⁴

test("calling fs returns a mocked call", async () => {
    //...
    const os = require("os")
    const HOME = os.homedir()
    fs.promises
        .readdir(HOME)
        .then((files: any) => files.find((file: any) => file === ".zshrc"))
        .then((file: any) =>
            fs.promises
                .access(`${HOME}/${file}`)
                .then(() => true)
                .catch(() => false)
        )
        .then((res: boolean) => expect(res).toBe(true))
}

This test passes on my machine precisely because I do in fact have a .zshrc in my home directory.

Conclusion

I haven’t yet decided whether or not you would ever want to actually take advantage of the fact that a mock set up with this factory pattern has access to the literal file system as well as the fake volume setup by memfs, however, seeing it in action really clarified how the mock is behaving in my mind, so I wanted to demonstrate it for future reference.

Stepping back from the specifics of this test, my experiences with nom, the issues I’ve had managing the code as its grown, including the horror of creating bugs even while addressing others are wonderful motivation to learning more about testing. Here’s to writing code that you can trust!

Footnotes

• ¹ nom is a CLI I’ve been working on for the past few months. I wrote an update on my progress recently, which can be found here.
• ² While I’ve not used any of Vadim’s file-system libraries before, his react-use collection of custom hooks is one of the toolboxes I reach for whenever I start a new React project.
• ³ I say simple now, however, it took me the better part of an hour to understand what was happening here.
• ⁴ I could have maintained the arrange, act, assert framework more cleanly had I converted this into async/await. Alas, I stuck with promise chains, which muddied the waters.