To upgrade or not to upgrade — what to do with dependencies?

The topic of March’s Salzburg DevOps Meetup was dependency management. Christian Köberl did a really good presentation on dependency management — focusing on how to automate dependency upgrades by using a dependency update bot. In the Q&A a really interesting discussion started. Why do I need to upgrade my dependencies?. The majority of arguments for upgrading dependencies were either “Because you have to”, “Because otherwise, you fall behind too far” or “Because it improves the security of your system”. Since only the minority of dependency upgrades contain security fixes, this was not really satisfying. Especially since dependency upgrades take time so your employer or customer should see a return on this time investment.

Your employer or customer should see a return on investment on the time you spend on dependency upgrades.

At the end of the meetup, however, I had a really interesting conversation with a participant who is self-employed and working for various smaller businesses. I was intrigued by how he would “sell” dependency upgrades to his clients. Especially, because I felt by working with smaller businesses he’d need to make a good case on the return on their investment. In other words — what is the incentive for his customer to pay X Euros to upgrade React from 17.0.0 to 17.0.1.

His answer was really interesting. He only proposes dependency upgrades that fix security vulnerabilities, because it’s much easier for clients to see the value of the money they spend. And in addition, it is a good chance to get in touch with the client again. But he doesn’t propose any of the other updates — because why would they care? The systems are working as they should anyhow.

Of course, the context is different whether you are working on projects for smaller businesses or you are in a team that is working on a bigger, more long-term, product. But still there needs to be a reason why dependency upgrade bots like dependabot or renovate get more and more popular. So what is the return on investment for upgrading dependencies in those bigger, more long-term projects?

How expensive are dependency upgrades

There are a few factors which in general make dependency upgrades more or less expensive.

• Strengths of your automated test suite: The more trust you have in your automated test suite, the cheaper it gets for you to verify that the application is still working as expected after a dependency upgrade.
• Deployment Frequency & Cycle Time: The more often you release changes to production, the easier it is to pinpoint potential service impairments to a dependency upgrade.
• Mean Time to Recover: If a dependency upgrade impairs your service, how long does it take to recover? The shorter the MTTR the lower the risk when upgrading dependencies.

That being said, what are the reasons for upgrading dependencies?

Dependency upgrades that fix security vulnerabilities

This is the most obvious one. When a security vulnerability is revealed in a dependency, the incentive to upgrade to a version where the vulnerability is fixed is definitively high. There are good tools like renovate or dependabot which can help you identify vulnerable dependencies and open pull requests — or even automatically merge them — with version bumps.

However, not all dependency updates are as simple as bumping a version number and reinstalling dependencies. If it turns out to be more work it’s also worth evaluating whether the vulnerability is even exploitable. If it’s not exploitable the ROI for upgrading is that you are safe in the future if you’d use the dependency another way.

Dependency upgrades that introduce a change you are already waiting for

An obvious case is you are using a dependency that has a bug you needed to workaround. Or the new version is adding a feature you’ve previously built yourself. In this case, the ROI is the reduced complexity in your codebase — and also the reduced responsibility of your codebase, because you shift the responsibility of the feature/bugfix to the maintainer of the dependency.

Another reason may be that you’ve identified a performance bottleneck in your application which you’ve tracked back to a dependency. If the new version of this dependency mitigates this performance bottleneck the ROI is very obvious.

Another reason may be a better developer experience. Imagine, you are building a web app using preact 8 and you want to upgrade to preact X, because you feel like using preact hooks is a really good developer experience for reusing stateful logic. For those cases, the ROI is much harder to quantify.

You don’t know what you don’t know

You can use automated dependency upgrades to get notified when a new version of a dependency you use is available. You can take this chance to read through the changelog of the dependency to figure out if this version bump actually contains a relevant feature for you.

How high is the ROI in such cases? It may be higher for your “main” dependencies. Imagine, you are building an app with Angular, then you might want to stay up to date with the latest updates. Since it’s your “main” framework there might be a high chance that a new release includes a feature you didn’t know about, but you could use. But what about all those dependencies to linters, underscores, lodashes or build-plugins? What is the chance a new version includes a feature you didn’t know about and you could use? Or to be even more provocative — do you even read the changelogs of all dependencies you upgrade? What is the ROI of the time you invest in those upgrades?

I don’t want to make a case for a specific strategy on when or how often to upgrade your dependencies. But I fear in most cases the strategy of how to handle dependency upgrades wasn’t a conscious decision based on how much time we spend on it, what the risks and potentials are and what the return on investment is. But I feel it’s our responsibility as professionals to make sure there is a return on investment especially for the time we spend on maintaining applications.