Unless you were hiding under a rock, you probably heard about the “event-stream situation” on NPM this week. TLDR: the original maintainer of the
event-stream package was tired of maintaining it. He handed over the reins to a different developer, who promptly injected malware and released a new version.
Much of the ink spilled on this subject focused around open source expectations and who. was. responsible. for. this. happening. I don’t want to spend much time on this part of the discussion, so I’ll keep it short by saying the entitlement on display in that Github issue was remarkable. Imagine a scenario where someone gave $20 to everyone who wanted it. Some folks in that thread would be angry because it wasn’t $50.
What I do want to spend time talking about is why this happened. And why does it keep happening to Node/NPM?
“Vendoring” is for the Olds
Before Node and NPM, web applications didn’t usually have gobs of dependencies. You might have jQuery, a few plugins, and maybe a dozen or so other helper libraries. Every dependency in your app could cause bugs, break, have security problems, or be a pain to maintain and upgrade. You didn’t take a dependency unless you had to. Dependencies haven’t gotten magically safer in 2018. All these risks are still present, but some folks have forgotten the message.
There was also no package manager to install the dependency for you. If you wanted to use third party code, you pulled up the file in your browser, hit Save As, and put it in a
lib folder somewhere. It was sort of a pain, but it worked. We didn’t have a term for it 10 years ago, but these days we call it
And a lot of people really hate it. It’s cumbersome to include a dependency when you’re vendoring! You have to save it, include it in your build process, make sure it’s committed to source control etc. There’s no easy way to upgrade that dependency. There’s no easy way for that dependency to depend on other dependencies. It’s so archaic!
Thus, NPM was born.
One Package is Good, Four Hundred Must be Better
npm install on a new box and you’re off to the races (ha ha).
Since it was easy to create and use dependencies, the JS developer community said, “no one should ever have to write a left padding function again!” And lo, we entered the era of micropackages. These are packages of only a few lines that do the barest minimum of functionality. It’s taking the Unix philosophy of “each tool doing one thing well”, and perverting it to the extreme. But too many packages is part of the reason things like
event-stream keep happening.
An Example From TrackJS
/js/lib folder, and it worked. Every. Single. Time.
But recently we needed to build a new piece of UI that was heavily client side interactive. We turned to React. We have a very basic “app”. We are only drawing a small part of our UI with it, and the whole thing is a handful of components. And still there are 418 depdendencies in the
node_modules folder! Just to draw a tiny bit of HTML. And look at some of these….
is-arrayish? What’s the difference between
isobject? Every one of these is another avenue for bugs or security issues to creep in.
^ of Doom
The default, when including a new package with NPM, is to specify it with the
^ operator (most recent minor version.)
This means whenever someone does a fresh install of this application they will get the most recent version of React that has a major version of 16. It could be React
16.154.45. If the library in question is using semantic versioning, there should be no intentional breaking changes between minor versions. But, in practice, that can never be counted on. Additionally, bugs surface frequently between minor versions. And anyways, there’s no rule that says a package maintainer has to use semver. If I want to inject a bitcoin wallet snatcher in a point release, by golly I will.
The scariest part is that you will also be retrieving the latest version of that package’s required dependencies. Perhaps those have been updated too! And maybe in ways that will break your app. And those kind of bugs and breaks are extra painful to track down. The layers of indirection are enough to make you scream.
No One Knows You’re a Cryptominer on The Internet
The final act in the
The Grass is Greener… on Microsoft’s side?
Contrast the dependency situation in Node with that of .Net. Our TrackJS web app is built with ASP.NET MVC and is roughly 6 years old. It’s tens of thousands of lines of code. It does a tremendous number of things. And to accomplish all of it, we have just over 40 dependencies managed with Nuget (the NPM for .Net). Of those, 25 are written and maintained by Microsoft. Another dozen are written and maintained by the companies whose APIs they integrate with (Stripe, AWS etc). We have under 10 dependencies written by anonymous open source contributors in the entirety of our (large) .Net code base. And for those we do use, not one has child dependencies.
The difference here is staggering. A 6-year-old, massive by comparison, .Net app has 10 times fewer dependencies than a 1-month-old handful of basic React components. On top of that, 80% of the .Net dependencies are built by companies with incentives to keep quality high, and the bitcoin mining to a minimum.
A Brighter Future?
If you combine hundreds of dependencies, pulled in from random github repositories maintained by anonymous developers, with constant upgrades, you’re going to end up with problems like
event-stream. But things are going (slowly) in the right direction. As of NPM 5 there is a package.lock.json file created every time you install a new dependency. It’s basically the same thing as
npm shrinkwrap except it happens automatically. If you check the file in, and another developer does an
npm install on their box, in theory, they should get the exact same dependency tree as you, version for version.
is-array and an
is-arrayish? Maybe it’s better to copy and paste those one-liners off Stack Overflow? And maybe it’s better to use packages created and maintained by companies that have a profit motive to do so?
Until the developer culture changes, your app will continue to break for all manner of reasons. If you want instant notifications for when you app is having problems, give TrackJS a try!