People I’ve talked to have mixed feelings about the JavaScript ecosystem.
Some of them express how they are tired of the JavaScript fatigue. Others claim we live in a JavaScript Renaissance.
Some scoff that it’s not a real language. Others note that it’s the first mainstream language where casual users provide the language designers with insights from real-world use before features are implemented.
Some treat JavaScript as a compile target, while others form subcultures around particular ways of writing it. And it runs everywhere: on your laptop, in your phone, on a server, and soon, perhaps, in your fridge.
Unconstrained by a single vendor, the JavaScript ecosystem closely reflects human culture. It is inventive, incremental, messy, assimilating everything on its way, and ubiquitous.
I’ll be honest: I love the melting pot of JavaScript. And while there’s no denying that it’s harder for beginners now to get into it than it was for me five years ago, I believe there are a few things we can do to make it more approachable.
But first, let’s see how the JavaScript ecosystem came to be this way.
A wild toolchain appears
Traditionally, software platforms were cultivated by the corporate gatekeepers that recognized the importance of attracting third-party developers. Companies like Apple and Microsoft provided a language, compiler, and set of SDKs for their platform.
There was usually one preferred way to talk to a database, one way to draw the UI, one way to use the network. You could step outside the paved path if you wanted to, but it was cold and lonely there. This brought stability but stifled innovation.
The web has changed this. While the standards committees spent years arguing about the future of a document-centric web, developers realized that the advantages of a cross-platform delivery mechanism for rich interactive apps is worth the pain of creating them with no SDK, module system, library, or compiler. After all, they could write their own. And they did.
Innovations in the browser JavaScript engines made rich client-side web apps possible. Node.js relied on those innovations to run JavaScript on the server and pioneered a JavaScript module system. Client-side apps were in dire need of a module system, too. Tools implementing a similar module API for the client-side apps appeared and started growing in popularity.
The server-side and client-side JavaScript ecosystems continued to enrich each other. Many modules that people wanted to share were environment-agnostic, so both communities eventually converged on using the Node package ecosystem. People often joke about it, but the fact is that the npm registry is the largest code-sharing platform in the human history.
Tools compensating for the lack of an “official” JavaScript SDK were often created by small and independent developer communities. Their authors were application developers who decided to educate themselves about parsers, source code manipulation, static analysis, optimization, and other topics that traditionally were in the domain of academics and vendor SDK providers.
Tool building became more common in JavaScript than in other communities I’ve been a part of. There was a sense that it was okay to be an amateur, because most people were. Once you got comfortable with building apps, you might as well start building tools for yourself and others. You didn’t have to be a compiler engineer to contribute to Babel.
It helped that JavaScript modules are distributed as source code, so you can always peek under the hood. Since npm places the dependencies right under your project directory, you are always a click away from the code that makes your app tick. To me, it felt like installing code made that code my responsibility. I could help fix bugs and make the tools I relied on better.
And yet…
The dependency iceberg
Today the prospect of looking at the dependencies is daunting to many developers. In a typical JavaScript project, there are so many! And since npm now flattens them, you are immediately exposed to all the transitive dependencies, too.
I think this might be a healthy thing, as relying on strangers is not free. It is up to us to figure out a sustainable way to support open-source projects. The current model of unpaid volunteers maintaining the infrastructure used by top companies is not holding up. Your node_modules folder is that realization, staring you in the face.
Webpack and Vue show that some popular projects can reach their funding goals, but it takes a lot of effort. Important lower-level projects with less marketing glitz can’t compete with that. I don’t have an answer to this, but I urge you to support the projects you rely on, whether through money, code or documentation contributions, or activities like issue triage.
However, my biggest concern about the JavaScript tooling is not the amount of code we rely on. Code that is not essential can be removed, packages replaced, and polyfills deleted when equivalent features land in our runtime environments. We can tighten up the unnecessary abstractions.
Some people say all build tools are unnecessary and should be thrown out of the window as browsers evolve, but I can’t agree with this. I’m happy to have a JavaScript parser in my build-time dependencies because it powers a static analysis tool that finds bugs in my app. And while some older tools outlive the need for them, new tools that make me more productive fill the gap.
So no, it’s not the dependency iceberg itself that is worrying me.
It’s the proliferation of configuration options.
Our tools make me fat
As tool builders we are tempted to make everything configurable. Every GitHub feature request is telling us that if we add this tiny option, this person would be able to use our tool! And so we keep adding those options, until the configuration becomes so complex that people write books about it.
Some tools try to offer the same flexibility while avoiding configuration. They embrace the Unix philosophy: They are simple programs meant to be chained together. However, in my experience, the order and manner in which they are wired together is crucial. Instead of learning a configuration format you have to learn their internals—or, more likely, how to work around them with hacks. And since there’s no abstraction at the top, nothing tells you where you messed up if you make a mistake.
Kathy Sierra wrote a terrific post called “Your app makes me fat” four years ago. I can’t stop thinking about it. You should absolutely read it, but here is the piece that hits home for me:
If your UX asks the user to make choices, for example, even if those choices are both clear and useful, the act of deciding is a cognitive drain. And not just while they’re deciding. … Even after we choose, an unconscious cognitive background thread is slowly consuming/leaking resources, “Was that the right choice?” … If our work drains a user’s cognitive resources, what does he lose? What else could he have done with those scarce, precious, easily-depleted resources? Maybe he’s trying to stick with that diet. Or practice guitar. Or play with his kids.
As tool builders, we often don’t clearly recognize this price.
Whenever we add a configuration flag that is useful for five percent of our users, we should consider its wider human implications. Is it worth the cognitive drain it creates for the 95 percent? How about for the five percent it was intended to help, if they forget what it does or doubt their decision?
Will it paralyze people with choice? Can we make extra effort to make the default option good enough for everyone? If we have to add configuration, can we reveal it incrementally so that the users aren’t exposed to it until they’re ready? Will they understand it, or will they give up and learn to copy and paste random options until something works?
If you, like me, are working on JavaScript tools, you’re a gatekeeper to the largest programming community in the world. This is scary. When we screw up and print an incomprehensible error message, somebody somewhere decides that they’re not cut out for programming. When they can’t correctly configure a project, we lose them before they even get started.
There are a few strategies we can follow to make JavaScript tools friendlier. None of these are new ideas, but I find that they’re consistently underapplied in the JavaScript ecosystem. Projects like Ember and Elm get this right, and we can all learn from them. Here’s my take on what’s missing.
Configuration should not stand in the way of getting started
A tool should work with (almost) no configuration. It is usually possible to pick sane defaults for the vast majority of options. If there is no immediately obvious default, you can resort to either a heuristic or a convention, or a mix of both, as long as you offer some way for the user to know what’s happening.
If you feel conflicted, don’t be afraid to make an arbitrary choice and stick with it. If it truly doesn’t matter, this is better than pushing this responsibility onto a newcomer. This can also reduce the mental cost of switching between projects, as they will often use the same default settings.
Resist adding more configuration than absolutely necessary
If this is a matter of opinion, don’t be afraid to take a hard stance on some issues, at least for a while. If you know that a large fraction of potential users might never adopt your tool because of the amount of configuration necessary, you can eventually relax the restrictions, but don’t give up right away.
Keep in mind that configuration does not just add cognitive load to your users’ lives, but to yours as well. Every new option exponentially increases the potential bug surface area. It’s how projects go from stable to fragile and bug-ridden. And it is much better to have some people adopt a competing project than for you to burn out putting out fires in yours.
Disclose advanced features progressively
If some feature requires a configuration option to work, you can delay asking for it until the user actually attempts to use that feature. This makes the learning curve less steep and helps them understand both why that option is necessary and to which feature it relates.
Mind your output
When you spend months looking at your tool’s output, you know which parts are meaningful, but a new user doesn’t. Their eyeballs nervously scan the error messages, failing to find the part that is relevant to fixing them. Print enough noisy warnings or unactionable errors, and you forever lose their trust.
There are many ways to improve the tool output: Only print actionable information, help users find mistakes, avoid jargon, use white space and colors where appropriate, and don’t forget about accessibility.
Create reusable toolboxes
If you often find yourself starting projects by cloning your last project because writing configuration files and setting up build tools is too daunting, you might be working on the wrong level of abstraction. Instead of using a few lower-level tools directly, consider wrapping them into a higher-level “toolbox” that you can share between different projects.
Yes, I really am proposing to solve the problem of too many tools with another tool. This is not a joke. Hear me out. I think this works because we are constraining the configuration surface rather than widening it.
Toolboxes, not boilerplates
Instead of maintaining the configuration and the build dependencies in every project, you can create a single “toolbox” package that provides a curated experience on top of the tools you already use.
We are using this approach in Create React App. Many think it’s a boilerplate generator, but it’s a little different. Even though it generates a project with a bundler, a linter, and a test runner, it hides them behind a single package that we maintain, and it works out of the box with no configuration. This helps us ensure that the tools work well together, their versions are always compatible, and our users don’t have to configure anything to get started.
When the new versions of the underlying tools come out, we update them on our side, tweak the configuration if necessary, and release an update to our shared dependency that everybody can upgrade to with a single command. I have heard from multiple people that even though they don’t use Create React App itself, they have adopted a similar strategy at their companies, centralizing the frontend tooling in one package.
Of course, some projects really need project-specific configuration. In Create React App, we solve this with an approach called “ejecting,” pioneered by Enclave. If you run the “eject” command, the configuration files and the underlying build dependencies get copied directly into your project. Now you can customize everything you want, but you don’t get the upstream updates since you have essentially forked the tool environment into your project. Ejecting has its downsides, but it lets you choose whether you want to maintain your own per-project tooling setup or not.
I find reusable toolboxes to be an interesting way to make JavaScript tooling approachable. They add some unfortunate indirection, but they also offer a powerful way to reimagine how our tools can work together if we structure a cohesive experience with intention instead of leaving it to chance.
Of course, this doesn’t mean we should stop making lower-level tools more user-friendly. We can improve the ecosystem in bottom-up and top-down directions at the same time.
You can help
Even if you don’t consider yourself a tool maker, I encourage you to think about these strategies and how they could apply to the projects you use. Is there an error message that’s too obscure? A configuration option missing a sane default? A sentence in the documentation you could improve? A missing higher-level tool?
If the project accepts issues, you can file an issue and describe your proposal, or you could just send a pull request. As maintainers, we often become oblivious to the usability issues in our projects because we spend so much time with them.
If you’re a maintainer, consider volunteering for an initiative like Codebar. Talk to the beginners, see how they’re using your tools and where they stumble. The effort you put in to the newcomer experience can make a real difference in people’s lives.
If you’ve recently been bitten by a JavaScript tooling problem, the road ahead might seem daunting. But I’ve seen projects do 180-degree turnarounds, going from usability nightmares to delightful tools in just a few months.
Through high-quality feedback and incremental improvements today, we can make JavaScript tooling more approachable for the person who tries it tomorrow.
I can’t wait to see what they create with it.