If you’ve ever created an API service, you’ve probably had to consider versioning—even if it was just to put “v1” somewhere in your configured URL paths. Versioning is a strategy we can use to label and manage unique states of our software as it changes over time. If we decide to change something, we can provide the updated software through a newly labeled version that’s completely isolated from the original. This makes common changes, such as renaming a component, easy for users to handle. Anyone uninterested in the change can safely ignore it, and anyone who wants to use the new name for the component can do so.
Versioning allows developers to write code against specific states of software, which is incredibly valuable. In this article, I’ll explore the issues developers must understand in order to manage API changes using versioning, specifically when it comes to web APIs. Since we want to provide the greatest benefit to API users without requiring significant work on their end, I’ll also dig into what it means to make backward-compatible changes. (It’s not as simple as it might seem!) I’ll look at why backward compatibility is dependent on how and where the API is used, as well. Finally, I’ll explore a few potential versioning strategies for web APIs, as well as the benefits and drawbacks of each.
Versioning to manage change
Versioning is ultimately a way to come to grips with change. Software is rarely static; even if no one is actively making changes to a software package, compiler optimizations and the like will change the underlying bytecode in potentially unpredictable ways. Our challenge becomes how best to manage that inevitable change.
It’s a bit of a balancing act. On the one hand, we want to create lots of amazing functionality and make it available to everyone who might want to use it. (And even if we don’t want to make lots of changes, we might be forced to for a variety of reasons, such as security patches.) On the other hand, we don’t want all this amazing new functionality—not to mention necessary changes—to create extra work for the people who’ve invested time and effort into integrating with our software. And when extra work is absolutely necessary, we don’t want that to come as a shock—to us or to anyone using the API.
If the goal is stability above all else, the safest option is to deploy new versions for every new change. However, any user interested in these changes would need to explicitly choose to use the newer version, every time. If your software is a shared library of some sort, that means users would have to download the new package and add it to their project. If it’s a web API, users would need to update their code to point to the new version or use a new client library. And while versioning for any type of software has its challenges, versioning web APIs comes with a unique and particularly tricky set, since those using the API are not in control of it. The developers building and designing the API therefore have a much stronger duty of care to their users.
Versioning for web APIs
Web APIs are interfaces that expose functionality over a remote network, often relying on HTTP as the underlying transport protocol. The remote nature of web APIs means clients can access functionality while the code that powers that functionality remains on a central server. This is an impressive feat, making powerful hardware accessible with a simple remote API call. But it also makes versioning more complicated.
The complication arises from the network divide between the user of the web API (the client) and the code that defines the web API and handles incoming client requests (the server). The client is at the mercy of the server—if, for instance, the server is shut down for maintenance, the client can’t do anything about it. This also means that if the server were to stop supporting an older version of the web API, clients that rely on that version go kaput; their code would just stop working. These externalities make finding the right versioning cadence a delicate proposition.
But the potential disaster of relying on a remotely managed service comes with a silver lining: The same power we as API designers have to disrupt remote clients can be used to improve their experience, with no effort on their end. Imagine if we were to deploy new hardware and code optimizations that allow an API request that used to take 10 seconds to take only 100 milliseconds. If we were to ship this update via a shared library, those using the software would need to manually upgrade to this new, faster version of the package. But with a web API, we could deploy it in place.
What changes should we consider safe for this auto-magic deployment? If someone’s code was written to depend on the request taking a full 10 seconds, your change to a faster version would break those assumptions—and their code. Is that your mistake or theirs? While these might not be major considerations for shared libraries and other shared code (because the user can control the upgrade), they’re critical for web APIs.
What is compatibility?
When it comes to web APIs, a client and a server are considered compatible when both sides can communicate with one another—but that’s table stakes for a web API. The real compatibility question—that of backward compatibility—arises when we want to deploy changes to the API service that don’t require any effort on the client’s side. These changes are backward compatible if the existing client code continues to function as it did before.
For API designers, it comes down to control. Unless we own the client code that has been written to use a web API, which is extremely unlikely, there’s not much stopping that code from relying on nuances (such as request latency) or accidents (such as bugs) that were never intended to be integral to the API. When even innocuous changes can be breaking or backward incompatible, how do we know what’s safe to deploy?
Defining backward compatibility
Some changes are obviously backward incompatible. For example, renaming anything or adding new preconditions, like a required field, will break any code that talks to the web API. But many other changes might seem safe—and turn out to be anything but. It’s just not possible to make foolproof rules for compatibility that will work for everyone. Rules make a lot of assumptions, not all of which will apply to every situation. Instead, you have to look at what your API does, who’s using it, and how those users define compatibility. With that in mind, you can evaluate what’s reasonable.
What if we added 30 new optional fields to an API response, for example? In theory, adding new fields or resources to an API shouldn’t break any existing code. In reality, that’s far from guaranteed. If our API is used for data processing, we can assume most clients’ machines have ample resources. But what if our API is meant to be consumed by small IoT devices deployed across a city? These devices typically have very limited compute, storage, and memory resources available, and adding 30 new fields, even if they’re optional, may cause them to run out of memory and crash. Suddenly, a change we assumed would be backward compatible could require new hardware to be deployed across an entire city.
To complicate matters further, what about all the updates you’re more or less obligated to make, like security patches, bug fixes, and changes required by law? Are they backward compatible? If a particular behavior was never intended in the first place, does a change to that behavior, which might break existing client-side code, constitute a breaking change? What if a bug has serious security implications? If the API is for a bank, this might be considered compatible. If it’s for a social network, maybe not.
What about performance-based improvements or regressions? Should the timing of API requests and responses hold firm, or can they fluctuate from one request to another? What about changing the default values over time? The answers to these questions will depend on the business context in which your API lives. There are no hard and fast rules because every company, and every product within a company, will have different engineering principles and different customers, with different expectations to match.
Strategies for web API versioning
Given the complexities of compatibility, what’s the most effective way to version a web API so that client code continues to work when new functionality is folded into an existing version? That is, how far can we push the versioning envelope? Strategies for versioning web APIs exist on a spectrum, from requiring no work at all to requiring significant work on the part of the user. Let’s explore three common versioning options and how they can suit different use cases.
Perpetual stability
At one end of the spectrum is perpetual stability. This strategy is simple: Never break anything. Ever. This means you take into consideration everything you know about your customers and err on the side of compatibility above all else. When it comes to ambiguous issues like whether to fix a bug or make something faster, your policy is to leave things as they are. If your customers are resource-sensitive, you might add new fields sparingly. You might also require users to opt in to new fields by specifying which fields you want returned in every request.
This doesn’t mean you can never deploy incompatible changes. It does mean the bar for what’s considered compatible is very high, and anything that doesn’t meet that high bar goes into an entirely new version. For example, performance optimizations on version one might be deployed as version two instead; 10 years from now, version one will continue to function and operate exactly as it does today. Client code would need to be updated to version two in order to benefit from these optimizations, but that’s the point: If customers want new features, they have to be willing to invest the effort to get them.
This strategy can be difficult to adhere to when it comes to security patches, however. It sucks for an API to be unstable and break client code, but it’s worse to break laws. Even with a strategy of perpetual stability, deploying security updates across all versions is the better choice.
Agile instability
On the other end of the spectrum is complete instability. This strategy is equally simple: Everything might break, so be prepared. In practical terms, this means you have just two live versions at any given time: a stable version and a preview of what’s to come. At some point, the stable version is deprecated and disabled, and what was once the preview version becomes the “stable” version. This would ideally happen on a regular schedule (e.g., every six months) but might also be an ad-hoc event. Either way, it’s important to notify users of the change in status: Send an email notification, and, more importantly, ensure API responses include a header specifying the deprecated state and impending deletion date of the version. This goes on as long as there are new backward-incompatible changes that merit a new version.
| DATE | VERSION 1 | VERSION 2 | VERSION 3 | VERSION 4 |
|---|---|---|---|---|
| Jan 1 | Stable | Preview | ||
| Mar 1 | Deprecated | Stable | Preview | |
| May 1 | Deprecated | Deprecated | Stable | Preview |
| Jul 1 | Deleted | Deprecated | Deprecated | Stable |
Here, the bar for what’s considered backward compatible is completely up to you as the API designer. While the standard no-nos exist—don’t rename things, don’t remove things, don’t start returning error codes for previously successful API calls—users are force-upgraded over time to the new version. It might make sense to simply push all changes into the next preview version—after all, it’ll be the stable one soon enough, so there’s no point rushing things down the pipeline unless they’re truly urgent. If the users of a web API are very active and willing to invest frequent effort into integration, this strategy balances the need to push out new functionality, fixes, and improvements against the burden of managing many different versions simultaneously.
SemVer for web APIs
Somewhere toward the middle of the spectrum is semantic versioning (SemVer) for web APIs. It’s a way of using three separate version numbers to convey the types of changes users can expect: major, minor, and patch (formatted as <major>.<minor>.<patch>). Changes in the major version number indicate a backward-incompatible change; those in the minor version number indicate a backward-compatible change; and those in the patch version number indicate a bug fix or other noticeable change.
Put simply: Backward-incompatible changes are deployed in the next major version, and the first number ticks up accordingly. Backward-compatible changes are deployed as part of the current major version, and count toward the next minor version. So, for example, if the current version is 1.0.0, a backward-compatible change would be deployed as version 1.1.0 and the previous version would continue to operate as usual. In these cases, users would access a given API version by specifying an exact version string in the URI requested (e.g., GET /v1.1.0/books/).
In practical terms, this strategy can provide the best of both worlds to end users, but, as with the perpetual stability strategy, it comes at the cost of more maintenance. (The main difference is one of magnitude.) The biggest benefit is that users are able to choose exactly which version they want to rely on. More change-averse users can stick with a specific version (for example, 1.2.3), while the more change-friendly can pin to a specific major version and adopt the latest available option, since all changes inside a major version should be compatible even as minor and patch version numbers increase. The obvious downside is the maintenance headache: You’ll be responsible for running and maintaining what might be many, many versions. (In other words, you might have lots of binaries deployed for each supported version, or lots of gatekeeping feature flags in a single service binary.) To keep this manageable, you’ll need a deprecation policy in which older versions are eventually turned down and removed.
Change is complicated
The bottom line is that versioning and compatibility are complex. Surprise!
It’s crucial you consider who’s using your API and proactively decide what balance you want to strike between stability and ease of adopting new functionality. For example, a strategy of agile instability would work terribly for IoT devices, which would need to be updated far too frequently. If you’re able to manage many versions, as in the agile instability and SemVer strategies, microservice architecture fits much more closely with lots of versions running simultaneously. If your API caters to users who have trouble updating their client code, you should probably err on the side of stability, setting a higher bar for what can be updated in a given version, as well as how long that version is maintained. Versioning and compatibility exist on a spectrum: It’s up to you to decide what makes the most sense for your API and those using it.
This article is based on chapter nine, “Versioning and Compatibility,” from API Design Patterns by JJ Geewax.