`nom`: project check-in, october 2020

I’ve been working on nom, a small CLI for managing markdown notes (which ultimately become the fodder for this blog) since late July. Three months later (+/-) seems like a reasonable time to step back and check in on how things are going. While I’m at it, I thought I’d explore some of what I’ve learned as well as discuss my path forward, because as will become clear, I’ve hit a bit of an impasse.

Lessons

When I began working on nom, I had a pretty decent idea of what I was shooting for: a CLI that would automate much of the boiler plate that goes into creating new notes for my blog with some handy additions around managing drafts. Beyond that, however, I didn’t know what I was getting into. I’d never written a CLI before. I may have tried to experiment with interactive scripts, but never anything robust. I’ve also managed to write mostly functional code in the past.

Writing nom has made a few things clear:

1. Interface Design Is Hard (And Fun)
2. Abstractions Aren’t Free
3. Testing Is Critical

Let’s dive into each of these.

Interface Design Is Hard (And Fun)

When discussing a CLI, the interface is how the user interacts with the application - whether by invoking a command and passing in flags or responding to the prompts.

The more I experiment with writing a CLI, the more I am coming to understand just how hard it is to get interfaces right. The best solution I’ve come up with so far is dogfooding. There are just too many aspects that I don’t appreciate until I try to use it and that once I do are extraordinarily clear.

Abstractions Aren’t Free

Speaking of clarity - one thing that’s become very clear over time as I’ve worked on nom is that abstractions are not free. In fact, the wrong abstraction is a huge tax on future development. As I mentioned, when I first started nom, I had a decent idea of what I wanted it to do. What I had no idea about was the how. Even worse, I didn’t take time to think through how the commands might interact.

The result was a mess of iterative designs that built on one another and have become more and more unwieldy over time. I’ve gone back to the drawing board and plan to refactor a lot of the mistakes I’ve made with a more cogent framework.

To make this example more concrete: when I started I didn’t realize that I would have core concepts of Configurations, Contents (i.e., a roster of all the notes), and Notes (i.e., the notes themselves). As a result, I implemented multiple different ways of accessing the configurations. Each command had a different way of reading notes. It was a mess.

Testing Is Critical

I love testing… in theory. I remember when I was a product manager, I would write my acceptance criteria from a behavioral perspective and I would encourage my engineers to write tests. Of course, I had no idea what was actually involved in writing tests at the time. I mention this only to highlight how easy it is to not write tests even when you think it’s a good idea (which I do!). I suspect that’s because in many ways tests are like insurance: they’re expensive and you never need it until you need it.

So, I skipped (automated) testing completely. It didn’t take long for the problem with this approach to reveal itself (about 20 days based on the git log). From that point on, I started spending approximately equal time, if not more, addressing bugs as I did building features. Again, a git log can be damning evidence for the need for a more robust testing strategy: I just didn’t listen and tried to push through. Needless to say, that will nto be my approach going forward.

Next Steps

Speaking of going forward, what exactly are my next steps with nom? Well, development stalled because I couldn’t add new features without introducing new bugs. This has been useful in that I’ve learned a bunch about debugging node, but not useful in pushing nom forward.

So, I’m taking a step back and focusing on two areas:

1. Refactoring
2. Retooling

Refactoring

Joel Spolsky famously (to me?) wrote about rewriting software in Things You Should Never Do, Part I. The good news is that I’m not rewriting from scratch (not that I think Joel’s point really applies here: I’m not a public company, I don’t have users, and I never said I was out of alpha, as of this writing, my latest version is 0.0.4, so there was no expectation that things wouldn’t change). Almost all of the refactoring should be hidden from the user because the API’s not the problem. The problem is with the implementation, which fortunately, despite all of my abstraction problems has been hidden from the user. This means that I should be able to completely change the implementation and not affect the user in any meaningfully way (though I’m reserving that right as well for now).

The focus of the refactor is two fold:

1. Align my commands along more coherent fault lines so that my commands do not try to recreate a wheel each team
2. Test early, test often, test behavior, not implementation.

Each step of the way with this refactor, I intend to adopt a Red, Green, Commit approach (I don’t think I understand enough to try a test && commit || revert … yet). The expectation is that this will help me avoid future reversions, rip out the old code, and move forward confidently more quickly (even if I have to teach myself the fundamentals of testing first).

Retooling

Shiny object syndrome afflicts people of all colors and persuasions, though with the pace of change in technology, engineers seem to be particularly prone to it. I acknowledge that fully even as I suggest that part of the refactor will be to swap out some of the tools I’ve been using to build nom.

This is one of the fun parts of building side projects: exploring the ecosystem of tools to solve problems. nom is my first CLI. Before it, I didn’t understand process.argv or how to create interactive prompts. These past few months I’ve explored a number of tools and some of my early choices I wouldn’t make if I were starting again. Since I’m effectively rebooting the project, I have an opportunity to reevaluate all of those decisions.

The tools won’t be dramatically different, after all, the goal of nom is the same: a simple interface to manage notes. The APIs are also largely similar. The reason to retool then is to fill in some gaps in some cases (i.e. add a testing framework) and expand my understanding by seeing the differences of building with A instead of B which can only happen when I actually get experience building with A and B.

Conclusion

I appreciate the opportunities that having a side project like nom presents. By virtue of being totally self-driven, I have the chance to wear multiple hats - not only am I lead engineer and architect, but I’m head of product, and, should I choose one day to promote its adoption, chief marketer. And, because ultimately I want nom to be useful for others, it requires that I take other people into consideration when designing it. Said another way, side projects can be an exercise in empathy!

On the other hand, there seem to be competing camps of thought. There’s the “you need to have side projects to succeed” camp and the equally vocal “you don’t need to code outside of work” camp. I’m not taking a side in that debate. Instead, I’m bringing them up because these views are simplifications. Some projects are fun and that’s why you work on them. Building nom is teaching me so much about Node, design principles, effective communication (both in writing but also in API design), and, going forward, testing. When it stops being fun, I take a break - which is a feature of side projects. Since they’re personal, there’s no need to slog through them if they’re not giving you anything.

I built am building nom for me. It’s supposed to be flexible enough to help others, but at its core its purpose is to automate a series of workflows that I use every day to write in my blog and take notes about life. The great part about that is that since I do this every day, I’m pretty motivated to get it work. It also means that I have the flexibility to experiment. That’s why I’m refactoring the entire project and beginning again with a test-first mindset.

So, three months in, where does that leave me? Well, I’ve learned a lot. I’ve explored a new problem domain. But most of all, I’m grateful. I’m grateful that I can work on nom. That I have the freedom to explore and learn. That I have colleagues and friends I can ask for assistance when I get stuck. nom’s been challenging in all of the best ways. It’s also been frustrating because every time something doesn’t go as expected it’s frustrating. I look at what some of my friends have accomplished in less time and I wonder if I’ll ever be as effective as they are, but then I remember where I was a year ago and I worry less. My journey is my own and how glorious it is! I truly am the luckiest person in the world.