books/The-API-Book
1
0
Fork 0
mirror of https://github.com/twirl/The-API-Book.git synced 2025-08-10 21:51:42 +02:00
Code Issues Releases Activity

proofreading

Browse Source
This commit is contained in:
Sergey Konstantinov
2023-05-31 10:34:24 +03:00
parent 991a6fd9f2
commit 89bb1aa895
1 changed files with 61 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
src/en/clean-copy/04-Section III. The Backward Compatibility/01.md
View File

@@ -2,124 +2,122 @@
As usual, let's conceptually define “backward compatibility” before we start.
Backward compatibility is a feature of the entire API system to be stable in time. It means the following: **the code that developers have written using your API continues working functionally correctly for a long period of time**. There are two important questions to this definition and two explanations:
Backward compatibility is a feature of the entire API system to be stable in time. It means the following: **the code that developers have written using your API continues to work functionally correctly for a long period of time**. There are two important questions regarding this definition and two explanations:
1. What does “functionally correctly” mean?
It means that the code continues to serve its function, i.e., to solve some users' problems. It doesn't mean it continues working indistinguishably from the previous version: for example, if you're maintaining a UI library, changing functionally insignificant design details like shadow depth or border stroke type is backward-compatible, whereas changing the sizes of the visual components is not.
It means that the code continues to serve its intended function which is to solve users' problems. It does not necessarily mean that it continues to work indistinguishably from the previous version. For example, if you're maintaining a UI library, making functionally insignificant design changes like adjusting shadow depth or border stroke type would still be considered backward-compatible unlike changing the sizes of the visual components.
2. What does “a long period of time” mean?
From our point of view, the backward compatibility maintenance period should be reconciled with the typical lifetime of applications in the subject area. Platform LTS periods are decent guidance in most cases. Since the applications will be rewritten anyway when the platform maintenance period ends, it is reasonable to expect developers to move to the new API version as well. In mainstream subject areas (i.e., desktop and mobile operating systems) this period lasts several years.
From our point of view, the backward compatibility maintenance period should be aligned with the typical lifetime of applications in the subject area. Platform LTS (Long-Term Support) periods can serve as helpful guidelines in most cases. Since applications will be rewritten when the platform maintenance period ends, it is reasonable to expect developers to transition to the new API version as well. In mainstream subject areas such as desktop and mobile operating systems, this period typically spans several years.
From the definition becomes obvious why backward compatibility needs to be maintained (including taking necessary measures at the API design stage). An outage, full or partial, caused by an API vendor, is an extremely uncomfortable situation for every developer, if not a disaster — especially if they pay money for the API usage.
The definition makes it evident why maintaining backward compatibility is crucial, including taking necessary measures at the API design stage. An outage, whether full or partial, caused by an API vendor, is an extremely inconvenient situation for every developer, if not a disaster — especially if they are paying for the API usage.
But let's take a look at the problem from another angle: why the problem of maintaining backward compatibility exists in the first place? Why would anyone *want* to break it? This question, though it looks quite trivial, is much more complicated than the previous one.
But let's take a look at the problem from another angle: why does the problem of maintaining backward compatibility exist in the first place? Why would anyone *want* to break it? This question, though it may seem trivial, is much more complicated than the previous one.
We could say that *we break backward compatibility to introduce new features to the API*. But that would be deceiving: new features are called *“new”* for a reason, as they cannot affect existing implementations which are not using them. We must admit there are several associated problems, which lead to the aspiration to rewrite *our* code, the code of the API itself, and ship a new major version:
We could say that *we break backward compatibility to introduce new features to the API*. However, this would be deceiving since new features are called *“new”* for a reason — they cannot affect existing implementations that do not use them. We must admit that there are several associated problems that lead to the aspiration to rewrite *our* code, the code of the API itself, and ship a new major version:
* the codebase eventually becomes outdated; making changes, even introducing totally new functionality, becomes impractical;
* The codebase eventually becomes outdated making it impractical to introduce changes or even new functionality
* The old interfaces are not suited to accommodate new features. We would love to extend existing functionality with new properties, but we simply cannot
* Finally, with years passing since the initial release, we have gained a better understanding of the subject area and API best practices. We would implement many things differently now.
* the old interfaces aren't suited to encompass new features; we would love to extend existing functionality with new properties, but we simply couldn't;
These arguments can be summarized frankly as “API vendors do not want to support old code.” However, this explanation is still incomplete. If you're not planning to rewrite the API code to add new functionality or even if you're not planning to add it at all, you still need to release new API versions, both minor and major.
* finally, with years passing since the initial release, we have understood more about the subject area and API best practices, and we would implement many things differently.
**NB**: in this chapter, we don't make any difference between minor versions and patches. “Minor version” means any backward-compatible API release.
These arguments could be summarized frankly as “the API vendors don't want to support the old code.” But this explanation is still incomplete: even if you're not going to rewrite the API code to add new functionality, or you're not going to add it at all, you still have to ship new API versions, minor and major alike.
Let us remind the reader that [an API is a bridge](#intro-api-definition), a means of connecting different programmable contexts. No matter how strong our desire is to keep the bridge intact, our capabilities are limited: we can lock the bridge, but we cannot command the rifts and the canyon itself. That's the source of the problem: we can't guarantee that *our own* code won't change. So at some point, we will have to ask the clients to rewrite *their* code.
**NB**: in this chapter, we don't make any difference between minor versions and patches: “minor version” means any backwards-compatible API release.
Apart from our aspirations to change the API architecture, three other tectonic processes are happening at the same time: user agents, subject areas, and the erosion of underlying platforms.
Let us remind the reader that [an API is a bridge](#intro-api-definition), a meaning of connecting different programmable contexts. No matter how strong our desire to keep the bridge intact is, our capabilities are limited: we could lock the bridge, but we cannot command the rifts and the canyon itself. That's the source of the problems: we can't guarantee that *our own* code won't change. So at some point, we will have to ask the clients to rewrite *their* code.
##### The Fragmentation of Consumer Applications
Apart from our aspirations to change the API architecture, three other tectonic processes are happening at the same time: user agents, subject areas, and underlying platforms erosion.
When you shipped the very first API version, and the initial clients started using it, the situation was perfect. However, this perfection doesn't last, and two scenarios are possible.
#### the Fragmentation of Consumer Applications
1. If the platform allows for fetching code on-demand, like the good old Web does, and you weren't too lazy to implement that code-on-demand feature (in the form of a platform SDK, such as JS API), then the evolution of your API is more or less under your control. Maintaining backward compatibility effectively means keeping *the client library* backward-compatible. As for client-server interaction, you have freedom.
When you shipped the very first API version, and the very first clients started to use it, the situation was perfect. There was only one version, and all clients were using only it. When this perfection ends, two scenarios are possible.
This doesn't mean that you can't break backward compatibility. You can still make a mistake with cache-control headers or simply overlook a bug in the code. Additionally, even code-on-demand systems don't get updated instantly. The author of this book faced a situation where users deliberately kept a browser tab open *for weeks* to avoid updates. However, in general, you usually don't have to support more than two API versions — the latest one and the penultimate one. Furthermore, you may consider rewriting the previous major version of the library, implementing it on top of the actual API version.
1. If the platform allows for fetching code on-demand as the good old Web does, and you weren't too lazy to implement that code-on-demand feature (in a form of a platform SDK — for example, JS API), then the evolution of your API is more or less under your control. Maintaining backward compatibility effectively means keeping *the client library* backwards-compatible. As for client-server interaction, you're free.
2. If the code-on-demand feature isn't supported or is prohibited by the platform, as is the case with modern mobile operating systems, the situation becomes more severe. Each client effectively borrows a snapshot of the code that works with your API, frozen at the moment of compilation. Client application updates are scattered over time to a much greater extent than Web application updates. The most painful aspect is that *some clients will never be up to date*, due to one of three reasons:
It doesn't mean that you can't break backward compatibility. You still can make a mess with cache-control headers or just overlook a bug in the code. Besides, even code-on-demand systems don't get updated instantly. The author of this book faced a situation when users were deliberately keeping a browser tab open *for weeks* to get rid of updates. But still, you usually don't have to support more than two API versions — the last one and the penultimate one. Furthermore, you may try to rewrite the previous major version of the library, implementing it on top of the actual API version.
* Developers simply don't want to update the app, i.e., its development has stopped.
* Users do not want to get updates (sometimes because they believe that developers “spoiled” the app in new versions)
* Users cannot get updates because their devices are no longer supported.
2. If the code-on-demand feature isn't supported or is prohibited by the platform, as in modern mobile operating systems, then the situation becomes more severe. Each client effectively borrows a snapshot of the code working with your API, frozen at the moment of compilation. Client application updates are scattered over time to much more extent than Web application updates. The most painful thing is that *some clients will never be up to date*, because one of three reasons:
In modern times these three categories combined could easily constitute a significant portion (tens of percent) of the audience. This implies that discontinuing support for any API version could be a nightmare experience — especially if partners' apps continue supporting a broader range of platforms than the API does.
* developers simply don't want to update the app, i.e., its development stopped;
* users don't want to get updates (sometimes because users think that developers “spoiled” the app in new versions);
* users can't get updates because their devices are no longer supported.
If you have never issued an SDK, providing only server-side APIs, for example in the form of HTTP endpoints, you might think that the backward compatibility problem is mitigated, although your API is less competitive on the market due to the lack of SDKs. However, that's not what will happen. If you don't provide an SDK, developers will either adopt an unofficial one (if someone bothered to create it) or write a framework themselves, independently. The “your framework — your problems” strategy, fortunately or unfortunately, works poorly. If developers write low-quality code on top of your API, then your API itself is of low quality — definitely in the view of developers and possibly in the view of end-users if the API's performance within the app is visible to them.
In modern times these three categories combined could easily constitute tens of per cent of auditory. It implies that cutting the support of any API version might be a nightmare experience — especially if developers' apps continue supporting a more broad spectrum of platforms than the API does.
Certainly, if you provide stateless APIs that don't require client SDKs or can be auto-generated from the spec, these problems will be much less noticeable. However, they are not fully avoidable unless you never issue any new API versions. If you do, you will still have to deal with some fragmentation of users by API and SDK versions.
You could have never issued any SDK, providing just the server-side API, for example in a form of HTTP endpoints. You might think that the backward compatibility problem is mitigated (by making your API less competitive on the market because of a lack of SDKs). That's not true: if you don't provide an SDK, then developers will either adopt an unofficial one (if someone bothered to make it) or just write a framework themselves — independently. “Your framework — your problems” strategy, fortunately, or not, works badly: if developers write low-quality code atop your API, then your API is of low quality itself — definitely in the view of developers, possibly in the view of end-users, if the API performance within the app is visible to them.
##### Subject Area Evolution
Certainly, if you provide a stateless API that doesn't require client SDKs (or they might be auto-generated from the spec), those problems will be much less noticeable, but not fully avoidable unless you never issue any new API version. If you do, you will still have to deal with some fragmentation of users by API and SDK versions.
The other side of the canyon is the underlying functionality that you expose via the API. It is, of course, not static and evolves in the following ways:
#### Subject Area Evolution
* New functionality emerges
* Older functionality shuts down
* Interfaces change.
The other side of the canyon is the underlying functionality you're exposing via the API. It's, of course, not static and somehow evolves:
As usual, the API provides an abstraction to a much more granular subject area. In the case of our coffee machine API example, one might reasonably expect new machine models to emerge, which will need to be supported by the platform. New models often come with new APIs, making it challenging to ensure their adoption while preserving the same high-level API. In any case, the API's code needs to be altered, which may lead to incompatibility, albeit unintentionally.
* new functionality emerges;
* older functionality shuts down;
* interfaces change.
Let us also emphasize that vendors of low-level APIs are not always as committed to maintaining backward compatibility for their APIs (or any software they provide) as we hope you are. It is important to be aware that keeping your API in an operational state, which involves writing and supporting facades to the shifting subject area landscape, will be your responsibility, sometimes posing quite sudden challenges.
As usual, the API provides an abstraction to a much more granular subject area. In the case of our coffee machine API example, one might reasonably expect new machine models to pop up, which are to be supported by the platform. New models tend to provide new APIs, and it's hard to guarantee they might be adopted while preserving the same high-level API. And anyway, the code needs to be altered, which might lead to incompatibility, albeit unintentional.
##### Platform Drift
Let us also stress that vendors of low-level API are not always as resolute regarding maintaining backward compatibility for their APIs (actually, any software they provide) as (we hope so) you are. You should be warned that keeping your API in an operational state, i.e., writing and supporting facades to the shifting subject area landscape, will be a problem of yours, and sometimes rather a sudden one.
Finally, there is a third aspect to consider — the “canyon” you are crossing over with a bridge of your API. Developers write code that is executed in an environment beyond your control, and it evolves. New versions of operating systems, browsers, protocols, and programming language SDKs emerge. New standards are being developed and new arrangements are made, some of which are backward-incompatible, and there is nothing that can be done about that.
#### Platform Drift
Older platform versions contribute to fragmentation just like older app versions as developers (including the API developers) struggle to support older platforms. At the same time, users face challenges with platform updates. In many cases, they are unable to update their devices to newer platform versions since newer platform versions require newer devices.
Finally, there is a third side to the story — the “canyon” you're crossing over with a bridge of your API. Developers write code that is executed in some environment you can't control, and it's evolving. New versions of operating systems, browsers, protocols, and programming language SDKs emerge. New standards are being developed and new arrangements made, some of them being backwards-incompatible, and nothing could be done about that.
Older platform versions lead to fragmentation just like older app versions do, because developers (including the API developers) are struggling with supporting older platforms, and users are struggling with platform updates — and often can't get updated at all, since newer platform versions require newer devices.
The nastiest thing here is that not only does incremental progress in a form of new platforms and protocols demand changing the API, but also does a vulgar fashion. Several years ago realistic 3d icons were popular, but since then the public taste changed in a favor of flat and abstract ones. UI components developers had to follow the fashion, rebuilding their libraries, either shipping new icons or replacing the old ones. Similarly, right now the “night mode” feature is introduced everywhere, demanding changes in a broad range of APIs.
The most challenging aspect here is that not only does incremental progress, in the form of new platforms and protocols, necessitate changes to the API, but also the vulgar influence of trends. Several years ago realistic 3D icons were popular, but since then, public taste has changed in favor of flat and abstract ones. UI component developers had to follow the fashion, rebuilding their libraries by either shipping new icons or replacing the old ones. Similarly, the current trend of integrating the “night mode” feature has become widespread, demanding changes in a wide range of APIs.
#### Backwards-Compatible Specifications
In the case of the API-first approach, the backward compatibility problem gets one more dimension: the specification and code generation based on it. It becomes possible to break backward compatibility without breaking the spec (let's say by introducing eventual consistency instead of the strict one) — and vice versa, modify the spec in a backwards-incompatible manner changing nothing in the protocol and therefore not affecting existing integrations at all (let's say, by replacing `additionalProperties: false` with `true` in OpenAPI).
In the case of the API-first approach, the backward compatibility problem adds another dimension: the specification and code generation based on it. It becomes possible to break backward compatibility without breaking the spec (for example, by introducing eventual consistency instead of strict consistency) — and vice versa, modify the spec in a backward-incompatible manner without changing anything in the protocol and therefore not affecting existing integrations at all (for example, by replacing `additionalProperties: false` with `true` in OpenAPI).
The question of whether two specification versions are backward-compatible or not rather belongs to a gray zone, as specification standards themselves do not define this. Generally speaking, the “specification change is backward-compatible” statement is equivalent to “any client code written or generated based on the previous version of the spec continues working correctly after the API vendor releases the new API version implementing the new version of the spec.” Practically speaking, following this definition seems quite unrealistic for two reasons: it's impossible to learn the behavior of every piece of code-generating software there (for instance, it's rather hard to say whether code generated based on a specification that includes the parameter `additionalProperties: false` will still function properly if the server starts returning additional fields).
The question of whether two specification versions are backward-compatible or not belongs to a gray zone, as specification standards themselves do not define this. Generally speaking, the statement “specification change is backward-compatible” is equivalent to “any client code written or generated based on the previous version of the spec continues to work correctly after the API vendor releases the new API version implementing the new version of the spec.” Practically speaking, following this definition seems quite unrealistic for two reasons: it is impossible to learn the behavior of every piece of code-generating software out there (for instance, it's rather hard to say whether code generated based on a specification that includes the parameter `additionalProperties: false` will still function properly if the server starts returning additional fields).
Thus, using IDLs to describe APIs with all advantages it undeniably brings to the field, leads to having one more side to the technology drift problem: the IDL version and, more importantly, versions of helper software based on it, are constantly and sometimes unpredictably evolving. If an API vendor employs the “code-first” approach, i.e., generates spec based on the actual API code, the occurrence of backward-incompatible changes in the server code — spec — code-generated SDK — client app chain is but a matter of time.
Thus, using IDLs to describe APIs with all the advantages they undeniably bring to the field, leads to having one aspect of the technology drift problem: the IDL version and, more importantly, versions of helper software based on it, are constantly and sometimes unpredictably evolving. If an API vendor employs the “code-first” approach, meaning that the spec is generated based on the actual API code, the occurrence of backward-incompatible changes in the server code — spec — code-generated SDK — client app chain is only a matter of time.
**NB**: we incline to recommend sticking to reasonable practices, i.e., don't use the functionality that is controversial from the backward compatibility point of view (including the above-mentioned `additionalProperties: false`) and, while evaluating the safety of changes, consider spec-generated code behave just like a manually written one. If you still get into the situation of unresolvable doubts, your only option is to manually check every code generator with regards to whether its output continues working with the new version of the API.
**NB**: we recommend sticking to reasonable practices such as not using functionality that is controversial from a backward compatibility point of view (including the above-mentioned `additionalProperties: false`) and when evaluating the safety of changes, considering spec-generated code behaves just like manually written code. If you find yourself in a situation of unresolvable doubts, your only option is to manually check every code generator to determine whether its output continues to work with the new version of the API.
#### Backward Compatibility Policy
To summarize the above:
* you will have to deploy new API versions because of apps, platforms, and subject areas evolution; different areas are evolving at a different pace, but never stop doing so;
* that will lead to fragmenting the API versions usage over different platforms and apps;
* you have to make decisions that greatly affect how sustainable your API is in your customers' view.
To summarize the points discussed above:
* You will have to deploy new API versions because of the evolution of apps, platforms, and subject areas. Different areas evolve at different paces but never stop doing so.
* This will result in the fragmentation of the API versions across different platforms and apps.
* You have to make decisions that greatly affect the sustainability of your API from your customers' perspective.
Let's briefly describe these decisions and the key factors for making them.
Let's briefly describe these decisions and the key factors to consider while making them.
1. How often new major API versions should be released?
1. How often should new major API versions be released?
That's primarily a *product* question. A new major API version is to be released when the critical mass of functionality is reached — a critical mass of features that couldn't be introduced in the previous API versions, or introducing them is too expensive. In stable markets, such a situation occurs once in several years, usually. In emerging markets, new major API versions might be shipped more frequently, only depending on your ability to support the zoo of the previous versions. However, we should note that deploying a new version before the previous one was stabilized (which commonly takes from several months up to a year) is always a troubling sign to developers meaning they're risking dealing with the platform glitches permanently.
This is primarily a *product* question. A new major API version should be released when a critical mass of functionality is reached, meaning a critical mass of features that couldn't be introduced in the previous API versions or would be too expensive to introduce. In stable markets, this situation typically occurs once every several years. In emerging markets, new major API versions might be shipped more frequently, depending only on your ability to support the zoo of the previous versions. However, it is important to note that deploying a new version before stabilizing the previous one, which commonly takes several months up to a year, is always a troubling sign to developers, as it means they risk dealing with API glitches permanently.
2. How many *major* versions should be supported at a time?
2. How many *major* versions should be supported simultaneously?
*Theoretically*, all of them. *Practically*, you should look at the size of the auditory which continues using older versions, and develop some guidance on when the support ends.
*Theoretically*, all of them. *Practically*, you should look at the size of the audience that continues to use older versions and develop guidelines on when the support for those versions will end.
3. How many *minor* versions (within one major version) should be supported at a time?
3. How many *minor* versions (within one major version) should be supported simultaneously?
As for minor versions, there are two options:
Regarding minor versions, there are two options:
* if you provide server-side APIs and compiled SDKs only, you may basically not expose minor versions at all (see below); however, at some maturity stage providing at least two latest versions becomes a must.
* if you provide code-on-demand SDKs, it is considered a good form to provide an access to previous minor versions of SDK for a period of time sufficient enough for developers to test their applications and fix issues if necessary. Since minor changes do not require rewriting large portions of code, it's fine to align the lifecycle of a minor version with the app release cycle duration in your industry, which is usually several months in the worst cases.
* If you provide server-side APIs and compiled SDKs only, you may basically choose not to expose minor versions at all (see below). However, at some maturity stage, providing access to at least the two latest versions becomes necessary.
* If you provide code-on-demand SDKs, it is considered good practice to provide access to previous minor versions of the SDK for a period of time sufficient for developers to test their applications and address issues if necessary. Since minor changes do not require rewriting large portions of code, it is acceptable to align the lifecycle of a minor version with the app release cycle duration in your industry, which in the worst cases may comprise several months.
#### Keeping Several API Versions
In modern professional software development, especially if we talk about internal APIs, a new API version usually fully replaces the previous one. If some problems are found, it might be rolled back (by releasing the previous version), but the two builds never co-exist. However, in the case of public APIs, the more the number of partner integrations is, the more dangerous this approach becomes.
In modern professional software development, especially when talking about internal APIs, a new API version usually fully replaces the previous one. If any problems are found, it might be rolled back by releasing the previous version, but the two builds never coexist. However, in the case of public APIs, the more partner integrations there are, the more dangerous this approach becomes.
Indeed, with the growth of the number of users, the “rollback the API version in case of problems” paradigm becomes increasingly destructive. To a partner, the optimal solution is rigidly referencing the specific API version — the one that had been tested (ideally, at the same time having the API vendor somehow seamlessly fix security concerns and make their software compliant with newly introduced legislation).
Indeed, with the growth in the number of users, the “rollback the API version in case of problems” paradigm becomes increasingly destructive. For partners, the optimal solution is rigidly referencing the specific API version — the one that had been tested (ideally, while also having the API vendor seamlessly address security concerns and make their software compliant with newly introduced legislation).
**NB**. From the same considerations, providing beta (or maybe even alpha) versions of the popular APIs becomes more and more desirable as well, to make partners test the upcoming versions and address the possible issues in advance.
**NB**: based on the same considerations, providing beta (or maybe even alpha) versions of popular APIs becomes more and more desirable as well, allowing partners to test upcoming versions and address possible issues in advance.
The important (and undeniable) advantage of the *semver* system is that it provides the proper version granularity:
The important (and undeniable) advantage of the *semver* system is that it provides proper version granularity:
* stating the first digit (major version) allows for getting a backwards-compatible version of the API;
* stating two digits (major and minor versions) allows guaranteeing that some functionality that was added after the initial release will be available;
* finally, stating all three numbers (major version, minor version, and patch) allows for fixing a concrete API release with all its specificities (and errors), which — theoretically — means that the integration will remain operable till this version is physically available.
* Stating the first digit (major version) allows obtaining a backward-compatible version of the API.
* stating two digits (major and minor versions) guarantees that functionality added after the initial release will be available.
* Finally, stating all three numbers (major version, minor version, and patch) allows fixing a concrete API release with all its specificities (and errors), which — theoretically — means that the integration will remain operational until this version becomes physically unavailable.
Of course, preserving minor versions infinitely isn't possible (partly because of security and compliance issues that tend to pile up). However, providing such access for a reasonable period of time is rather a hygienic norm for popular APIs.
Of course, preserving minor versions indefinitely is not possible (partly because of security and compliance issues that tend to accumulate). However, providing such access for a reasonable period of time is considered a hygienic norm for popular APIs.
**NB**. Sometimes to defend the single accessible API version concept, the following argument is put forward: preserving the SDK or API application server code is not enough to maintain strict backward compatibility as it might be relying on some un-versioned services (for example, some data in the DB that are shared between all the API versions). We, however, consider this an additional reason to isolate such dependencies (see “[The Serenity Notepad](#back-compat-serenity-notepad)” chapter) as it means that changes to these subsystems might lead to the inoperability of the API.
**NB**. Sometimes to defend the concept of a single accessible API version, the following argument is put forward: preserving the SDK or API application server code is not enough to maintain strict backward compatibility as it might rely on some unversioned services (for example, data in the DB shared between all API versions). However, we consider this an additional reason to isolate such dependencies (see “[The Serenity Notepad](#back-compat-serenity-notepad)” chapter) as it means that changes to these subsystems might result in the API becoming inoperable.