mirror of
https://github.com/twirl/The-API-Book.git
synced 2025-05-31 22:09:37 +02:00
On Design Patterns
This commit is contained in:
parent
551b5a16f9
commit
a7becf1b80
Binary file not shown.
@ -639,7 +639,7 @@ ul.references li p a.back-anchor {
|
||||
</li>
|
||||
<li>the API must be consistent
|
||||
<ul>
|
||||
<li>while developing new functionality (i.e. while using previously unknown API entities) developers may write new code similar to the code they have already written using the known API concepts, and this new code will work.</li>
|
||||
<li>while developing new functionality (i.e., while using previously unknown API entities) developers may write new code similar to the code they have already written using the known API concepts, and this new code will work.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
@ -654,13 +654,13 @@ ul.references li p a.back-anchor {
|
||||
<p>We will discuss API lifecycle and version policies in Section II.</p><div class="page-break"></div><h3><a href="#intro-versioning" class="anchor" id="intro-versioning">Chapter 5. On Versioning</a><a href="#chapter-5" class="secondary-anchor" id="chapter-5"> </a></h3>
|
||||
<p>Here and throughout this book, we firmly stick to <a href="https://semver.org/">semver</a> principles of versioning.</p>
|
||||
<ol>
|
||||
<li>API versions are denoted with three numbers, i.e. <code>1.2.3</code>.</li>
|
||||
<li>API versions are denoted with three numbers, i.e., <code>1.2.3</code>.</li>
|
||||
<li>The first number (a major version) increases when backwards-incompatible changes in the API are introduced.</li>
|
||||
<li>The second number (a minor version) increases when new functionality is added to the API, keeping backwards compatibility intact.</li>
|
||||
<li>The third number (a patch) increases when a new API version contains bug fixes only.</li>
|
||||
</ol>
|
||||
<p>Sentences “a major API version” and “new API version, containing backwards-incompatible changes” are therefore to be considered equivalent ones.</p>
|
||||
<p>It is usually (though not necessary) agreed that the last stable API release might be referenced by either a full version (e.g. <code>1.2.3</code>) or a reduced one (<code>1.2</code> or just <code>1</code>). Some systems support more sophisticated schemes of defining the desired version (for example, <code>^1.2.3</code> reads like “get the last stable API release that is backwards-compatible to the <code>1.2.3</code> version”) or additional shortcuts (for example, <code>1.2-beta</code> to refer to the last beta release of the <code>1.2</code> API version family). In this book, we will mostly use designations like <code>v1</code> (<code>v2</code>, <code>v3</code>, etc.) to denote the latest stable release of the <code>1.x.x</code> version family of an API.</p>
|
||||
<p>It is usually (though not necessary) agreed that the last stable API release might be referenced by either a full version (e.g., <code>1.2.3</code>) or a reduced one (<code>1.2</code> or just <code>1</code>). Some systems support more sophisticated schemes of defining the desired version (for example, <code>^1.2.3</code> reads like “get the last stable API release that is backwards-compatible to the <code>1.2.3</code> version”) or additional shortcuts (for example, <code>1.2-beta</code> to refer to the last beta release of the <code>1.2</code> API version family). In this book, we will mostly use designations like <code>v1</code> (<code>v2</code>, <code>v3</code>, etc.) to denote the latest stable release of the <code>1.x.x</code> version family of an API.</p>
|
||||
<p>The practical meaning of this versioning system and the applicable policies will be discussed in more detail in the <a href="#back-compat-statement">“Backwards Compatibility Problem Statement”</a> chapter.</p><div class="page-break"></div><h3><a href="#intro-terms-notation" class="anchor" id="intro-terms-notation">Chapter 6. Terms and Notation Keys</a><a href="#chapter-6" class="secondary-anchor" id="chapter-6"> </a></h3>
|
||||
<p>Software development is characterized, among other things, by the existence of many different engineering paradigms, whose adepts sometimes are quite aggressive towards other paradigms' adepts. While writing this book, we are deliberately avoiding using terms like “method,” “object,” “function,” and so on, using the neutral term “entity” instead. “Entity” means some atomic functionality unit, like class, method, object, monad, prototype (underline what you think is right).</p>
|
||||
<p>As for an entity's components, we regretfully failed to find a proper term, so we will use the words “fields” and “methods.”</p>
|
||||
@ -756,7 +756,7 @@ Cache-Control: no-cache
|
||||
<p>Since our book is dedicated not to software development per se, but to developing APIs, we should look at all those questions from a different angle: why does solving those problems specifically require an API, not simply a specialized software application? In terms of our fictional example, we should ask ourselves: why provide a service to developers, allowing for brewing coffee to end users, instead of just making an app?</p>
|
||||
<p>In other words, there must be a solid reason to split two software development domains: there are vendors that provide APIs, and there are vendors that develop services for end users. Their interests are somehow different to such an extent that coupling these two roles in one entity is undesirable. We will talk about the motivation to specifically provide APIs instead of apps (or as an addition to an app) in more detail in Section III.</p>
|
||||
<p>We should also note that you should try making an API when, and only when, your answer to question (3) is "because that's our area of expertise". Developing APIs is a sort of meta-engineering: you're writing some software to allow other vendors to develop software to solve users' problems. You must possess expertise in both domains (APIs and user products) to design your API well.</p>
|
||||
<p>As for our speculative example, let us imagine that in the nearby future, some tectonic shift happened within the coffee brewing market. Two distinct player groups took shape: some companies provide “hardware,” i.e. coffee machines; other companies have access to customer auditory. Something like the modern-day flights market looks like: there are air companies, which actually transport passengers; and there are trip planning services where users are choosing between trip variants the system generates for them. We're aggregating hardware access to allow app vendors for ordering freshly brewed coffee.</p>
|
||||
<p>As for our speculative example, let us imagine that in the nearby future, some tectonic shift happened within the coffee brewing market. Two distinct player groups took shape: some companies provide “hardware,” i.e., coffee machines; other companies have access to customer auditory. Something like the modern-day flights market looks like: there are air companies, which actually transport passengers; and there are trip planning services where users are choosing between trip variants the system generates for them. We're aggregating hardware access to allow app vendors for ordering freshly brewed coffee.</p>
|
||||
<h4>What and How</h4>
|
||||
<p>After finishing all these theoretical exercises, we should proceed right to designing and developing the API, having a decent understanding of two things:</p>
|
||||
<ul>
|
||||
@ -1080,7 +1080,7 @@ GET /sensors
|
||||
<li>call <code>POST /execute</code> physical API method, passing internal program identifier — for the first API kind;</li>
|
||||
<li>initiate runtime creation to proceed with the second API kind.</li>
|
||||
</ul>
|
||||
<p>Out of general considerations, the runtime level for the second-kind API will be private, so we are more or less free in implementing it. The easiest solution would be to develop a virtual state machine that creates a “runtime” (e.g. a stateful execution context) to run a program and control its state.</p>
|
||||
<p>Out of general considerations, the runtime level for the second-kind API will be private, so we are more or less free in implementing it. The easiest solution would be to develop a virtual state machine that creates a “runtime” (i.e., a stateful execution context) to run a program and control its state.</p>
|
||||
<pre><code>POST /v1/runtimes
|
||||
{
|
||||
"coffee_machine",
|
||||
@ -1158,14 +1158,14 @@ GET /sensors
|
||||
<li>there might be explicit proxying of calls down the hierarchy;</li>
|
||||
<li>there might be a cache at each level, being updated upon receiving a callback call or an event. In particular, a low-level runtime execution cycle obviously must be independent of upper levels, which implies renewing its state in the background, and not waiting for an explicit call.</li>
|
||||
</ul>
|
||||
<p>Note what happens here: each abstraction level wields its own status (e.g. order, runtime, sensors status), being formulated in subject area terms corresponding to this level. Forbidding the “jumping over” results in the necessity to spawn statuses at each level independently.</p>
|
||||
<p>Note what happens here: each abstraction level wields its own status (e.g., order, runtime, sensors status), being formulated in subject area terms corresponding to this level. Forbidding the “jumping over” results in the necessity to spawn statuses at each level independently.</p>
|
||||
<p>Let's now look at how the order cancel operation flows through our abstraction levels. In this case, the call chain will look like that:</p>
|
||||
<ul>
|
||||
<li>a user initiates a call to the <code>POST /v1/orders/{id}/cancel</code> method;</li>
|
||||
<li>the method handler completes operations on its level of responsibility:
|
||||
<ul>
|
||||
<li>checks the authorization;</li>
|
||||
<li>solves money issues, i.e. whether a refund is needed;</li>
|
||||
<li>solves money issues, i.e., whether a refund is needed;</li>
|
||||
<li>finds the <code>program_run_id</code> identifier and calls the <code>runs/{program_run_id}/cancel</code> method;</li>
|
||||
</ul>
|
||||
</li>
|
||||
@ -1193,7 +1193,7 @@ The interim API level is needed to make this transition between different level
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<p>From a high-level point of view, canceling an order is a terminal action since no further operations are possible. From a low-level point of view, the processing continues until the cup is discarded, and then the machine is to be unlocked (e.g. new runtimes creation allowed). It's a task to the execution control level to couple those two states, outer (the order is canceled) and inner (the execution continues).</p>
|
||||
<p>From a high-level point of view, canceling an order is a terminal action since no further operations are possible. From a low-level point of view, the processing continues until the cup is discarded, and then the machine is to be unlocked (i.e., new runtimes creation allowed). It's a task to the execution control level to couple those two states, outer (the order is canceled) and inner (the execution continues).</p>
|
||||
</li>
|
||||
</ol>
|
||||
<p>It might look like forcing the abstraction levels isolation is redundant and makes interfaces more complicated. In fact, it is: it's very important to understand that flexibility, consistency, readability, and extensibility come with a price. One may construct an API with zero overhead, essentially just providing access to the coffee machine's microcontrollers. However using such an API would be a disaster for a developer, not to mention the inability to extend it.</p>
|
||||
@ -1204,7 +1204,7 @@ The interim API level is needed to make this transition between different level
|
||||
<p>What data flow do we have in our coffee API?</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>It starts with the sensors data, i.e. volumes of coffee / water / cups. This is the lowest data level we have, and here we can't change anything.</p>
|
||||
<p>It starts with the sensors data, e.g., volumes of coffee / water / cups. This is the lowest data level we have, and here we can't change anything.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>A continuous sensors data stream is being transformed into discrete command execution statuses, injecting new concepts which don't exist within the subject area. A coffee machine API doesn't provide a “coffee is being poured” or a “cup is being set” notion. It's our software that treats incoming sensor data and introduces new terms: if the volume of coffee or water is less than the target one, then the process isn't over yet. If the target value is reached, then this synthetic status is to be switched, and the next command is executed.
|
||||
@ -1253,7 +1253,7 @@ It is important to note that we don't calculate new variables out of sensor data
|
||||
<li>canceled.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>A <code>recipe</code> describes an “ideal model” of some coffee beverage type, e.g. its customer properties. A <code>recipe</code> is an immutable entity for us, which means we could only read it.</li>
|
||||
<li>A <code>recipe</code> describes an “ideal model” of some coffee beverage type, i.e., its customer properties. A <code>recipe</code> is an immutable entity for us, which means we could only read it.</li>
|
||||
<li>A <code>coffee-machine</code> is a model of a real-world device. We must be able to retrieve the coffee machine's geographical location and the options it supports from this model (which will be discussed below).</li>
|
||||
</ul>
|
||||
</li>
|
||||
@ -1272,7 +1272,7 @@ It is important to note that we don't calculate new variables out of sensor data
|
||||
</li>
|
||||
<li>Runtime-level entities.
|
||||
<ul>
|
||||
<li>A <code>runtime</code> describes a specific execution data context, i.e. the state of each variable. A <code>runtime</code> might be:
|
||||
<li>A <code>runtime</code> describes a specific execution data context, i.e., the state of each variable. A <code>runtime</code> might be:
|
||||
<ul>
|
||||
<li>initialized (created)</li>
|
||||
<li>checked for its status</li>
|
||||
@ -1600,10 +1600,10 @@ strpbrk (str1, str2)
|
||||
<p><strong>NB</strong>: sometimes field names are shortened or even omitted (e.g., a heterogenous array is passed instead of a set of named fields) to lessen the amount of traffic. In most cases, this is absolutely meaningless as usually the data is compressed at the protocol level.</p>
|
||||
<h5><a href="#chapter-11-paragraph-5" id="chapter-11-paragraph-5" class="anchor">5. Naming Implies Typing</a></h5>
|
||||
<p>A field named <code>recipe</code> must be of a <code>Recipe</code> type. A field named <code>recipe_id</code> must contain a recipe identifier that we could find within the <code>Recipe</code> entity.</p>
|
||||
<p>Same for primitive types. Arrays must be named in a plural form or as collective nouns, i.e. <code>objects</code>, <code>children</code>. If that's impossible, better add a prefix or a postfix to avoid doubt.</p>
|
||||
<p>Same for primitive types. Arrays must be named in a plural form or as collective nouns, e.g., <code>objects</code>, <code>children</code>. If that's impossible, better add a prefix or a postfix to avoid doubt.</p>
|
||||
<p><strong>Bad</strong>: <code>GET /news</code> — unclear whether a specific news item is returned, or a list of them.</p>
|
||||
<p><strong>Better</strong>: <code>GET /news-list</code>.</p>
|
||||
<p>Similarly, if a Boolean value is expected, entity naming must describe some qualitative state, i.e. <code>is_ready</code>, <code>open_now</code>.</p>
|
||||
<p>Similarly, if a Boolean value is expected, entity naming must describe some qualitative state, e.g., <code>is_ready</code>, <code>open_now</code>.</p>
|
||||
<p><strong>Bad</strong>: <code>"task.status": true</code><br>
|
||||
— statuses are not explicitly binary; also such API isn't extendable.</p>
|
||||
<p><strong>Better</strong>: <code>"task.is_finished": true</code>.</p>
|
||||
@ -1676,7 +1676,7 @@ const order = api.createOrder(
|
||||
…
|
||||
}
|
||||
</code></pre>
|
||||
<p>This practice makes the code more complicated, and it's quite easy to make mistakes, which will effectively treat the field in an opposite manner. The same could happen if some special values (i.e. <code>null</code> or <code>-1</code>) to denote value absence are used.</p>
|
||||
<p>This practice makes the code more complicated, and it's quite easy to make mistakes, which will effectively treat the field in an opposite manner. The same could happen if some special values (e.g., <code>null</code> or <code>-1</code>) to denote value absence are used.</p>
|
||||
<p><strong>NB</strong>: this observation is not valid if both the platform and the protocol unambiguously support special tokens to reset a field to its default value with zero abstraction overhead. However, full and consistent support of this functionality rarely sees implementation. Arguably, the only example of such an API among those being popular nowadays is SQL: the language has the <code>NULL</code> concept, default field values functionality, and support for operations like <code>UPDATE … SET field = DEFAULT</code> (in most dialects). Though working with the protocol is still complicated (for example, in many dialects there is no simple method of getting back those values reset by an <code>UPDATE … DEFAULT</code> query), SQL features operating defaults conveniently enough to use this functionality as is.</p>
|
||||
<p>If the protocol does not support resetting to default values as a first-class citizen, the universal rule is to make all new Boolean flags false by default.</p>
|
||||
<p><strong>Better</strong></p>
|
||||
@ -2085,7 +2085,7 @@ GET /records?sort_by=date_modified⮠
|
||||
</code></pre>
|
||||
<p>Sorting by the date of modification usually means that data might be modified. In other words, some records might change after the first data chunk is returned, but before the next chunk is requested. Modified records will simply disappear from the listing because of moving to the first page. Clients will never get those records that were changed during the iteration process, even if the cursor-based scheme is implemented, and they never learn the sheer fact of such an omission. Also, this particular interface isn't extendable as there is no way to add sorting by two or more fields.</p>
|
||||
<p><strong>Better</strong>: there is no general solution to this problem in this formulation. Listing records by modification time will always be unpredictably volatile, so we have to change the approach itself; we have two options.</p>
|
||||
<p><strong>Option one</strong>: fix the records ordering at the moment we've got the initial request, e.g. our server produces the entire list and stores it in the immutable form:</p>
|
||||
<p><strong>Option one</strong>: fix the records ordering at the moment we've got the initial request, i.e., our server produces the entire list and stores it in the immutable form:</p>
|
||||
<pre><code>// Creates a view based on the parameters passed
|
||||
POST /v1/record-views
|
||||
{
|
||||
@ -2116,11 +2116,11 @@ GET /v1/records/modified/list⮠
|
||||
</code></pre>
|
||||
<p>This scheme's downsides are the necessity to create separate indexed event storage, and the multiplication of data items, since for a single record many events might exist.</p>
|
||||
<h4>Ensuring the Technical Quality of APIs</h4>
|
||||
<p>Fine APIs must not only solve developers' and end users' problems but also ensure the quality of the solution, e.g. do not contain logical and technical mistakes (and do not provoke developers to make them), save computational resources, and in general implement the best practices applicable to the subject area.</p>
|
||||
<p>Fine APIs must not only solve developers' and end users' problems but also ensure the quality of the solution, i.e., do not contain logical and technical mistakes (and do not provoke developers to make them), save computational resources, and in general implement the best practices applicable to the subject area.</p>
|
||||
<h5><a href="#chapter-11-paragraph-15" id="chapter-11-paragraph-15" class="anchor">15. Keep the Precision of Fractional Numbers Intact</a></h5>
|
||||
<p>If the protocol allows, fractional numbers with fixed precision (like money sums) must be represented as a specially designed type like Decimal or its equivalent.</p>
|
||||
<p>If there is no Decimal type in the protocol (for instance, JSON doesn't have one), you should either use integers (e.g. apply a fixed multiplicator) or strings.</p>
|
||||
<p>If conversion to a float number will certainly lead to losing the precision (let's say if we translate “20 minutes” into hours as a decimal fraction), it's better to either stick to a fully precise format (e.g. opt for <code>00:20</code> instead of <code>0.33333…</code>) or to provide an SDK to work with this data, or as a last resort describe the rounding principles in the documentation.</p>
|
||||
<p>If there is no Decimal type in the protocol (for instance, JSON doesn't have one), you should either use integers (e.g., apply a fixed multiplicator) or strings.</p>
|
||||
<p>If conversion to a float number will certainly lead to losing the precision (let's say if we translate “20 minutes” into hours as a decimal fraction), it's better to either stick to a fully precise format (e.g., opt for <code>00:20</code> instead of <code>0.33333…</code>) or to provide an SDK to work with this data, or as a last resort describe the rounding principles in the documentation.</p>
|
||||
<h5><a href="#chapter-11-paragraph-16" id="chapter-11-paragraph-16" class="anchor">16. All API Operations Must Be Idempotent</a></h5>
|
||||
<p>Let us remind the reader that idempotency is the following property: repeated calls to the same function with the same parameters won't change the resource state. Since we're discussing client-server interaction in the first place, repeating requests in case of network failure isn't an exception, but a norm of life.</p>
|
||||
<p>If the endpoint's idempotency can't be assured naturally, explicit idempotency parameters must be added, in a form of either a token or a resource version.</p>
|
||||
@ -2189,7 +2189,7 @@ X-Idempotency-Token: <token>
|
||||
<li>clients tend to misunderstand the concept and either generate new tokens each time they repeat the request (which deteriorates the UX, but otherwise healthy) or conversely use one token in several requests (not healthy at all and could lead to catastrophic disasters; another reason to implement the suggestion in the previous clause); writing detailed doc and/or client library is highly recommended.</li>
|
||||
</ul>
|
||||
<h5><a href="#chapter-11-paragraph-17" id="chapter-11-paragraph-17" class="anchor">17. Avoid Non-Atomic Operations</a></h5>
|
||||
<p>There is a common problem with implementing the changes list approach: what to do if some changes were successfully applied, while others are not? The rule is simple: if you may ensure the atomicity (e.g. either apply all changes or none of them) — do it.</p>
|
||||
<p>There is a common problem with implementing the changes list approach: what to do if some changes were successfully applied, while others are not? The rule is simple: if you may ensure the atomicity (i.e., either apply all changes or none of them) — do it.</p>
|
||||
<p><strong>Bad</strong>:</p>
|
||||
<pre><code>// Returns a list of recipes
|
||||
api.getRecipes();
|
||||
@ -2392,8 +2392,8 @@ PATCH /v1/orders/{id}
|
||||
{ /* updates accepted */ }
|
||||
</code></pre>
|
||||
<p>This signature is bad per se as it's unreadable. What does <code>null</code> as the first array element mean — is it a deletion of an element or an indication that no actions are needed towards it? What happens with the fields that are not stated in the update operation body (<code>delivery_address</code>, <code>milk_type</code>) — will they be reset to defaults, or stay unchanged?</p>
|
||||
<p>The nastiest part is that whatever option you choose, the number of problems will only multiply further. Let's say we agreed that the <code>{"items":[null, {…}]}</code> statement means that the first element of the array is left untouched, e.g. no changes are needed. Then, how shall we encode its deletion? Invent one more “magical” value meaning “remove it”? Similarly, if the fields that are not explicitly mentioned retain their value — how would you reset them to defaults?</p>
|
||||
<p><strong>The simple solution</strong> is always rewriting the data entirely, e.g. to require passing the entire object, to replace the current state with it, and to return the full state as a result of the operation. This obvious solution is frequently rejected with the following reasoning:</p>
|
||||
<p>The nastiest part is that whatever option you choose, the number of problems will only multiply further. Let's say we agreed that the <code>{"items":[null, {…}]}</code> statement means that the first element of the array is left untouched, i.e., no changes are needed. Then, how shall we encode its deletion? Invent one more “magical” value meaning “remove it”? Similarly, if the fields that are not explicitly mentioned retain their value — how would you reset them to defaults?</p>
|
||||
<p><strong>The simple solution</strong> is always rewriting the data entirely, i.e., to require passing the entire object, to replace the current state with it, and to return the full state as a result of the operation. This obvious solution is frequently rejected with the following reasoning:</p>
|
||||
<ul>
|
||||
<li>increased requests sizes and therefore, the amount of traffic;</li>
|
||||
<li>the necessity to detect which fields are changed (for instance, to generate proper state change events for subscribers);</li>
|
||||
@ -2408,9 +2408,9 @@ PATCH /v1/orders/{id}
|
||||
<li>furthermore, the existence of the client algorithm for finding the fields that changed doesn't mean that the server might skip implementing it as client developers might make mistakes or simply spare the effort and always send all the fields;</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>finally, this naïve approach to organizing collaborative editing works only with transitive changes (e.g. if the final result does not depend on the order in which the operations were executed), and in our case, it's already not true: deletion of the first element and editing the second element are non-transitive;
|
||||
<li>finally, this naïve approach to organizing collaborative editing works only with transitive changes (i.e., if the final result does not depend on the order in which the operations were executed), and in our case, it's already not true: deletion of the first element and editing the second element are non-transitive;
|
||||
<ul>
|
||||
<li>often, in addition to sparing traffic on requests, the same concept is applied to responses as well, e.g. no data is returned for modifying operations; thus two clients making simultaneous edits do not see one another's changes.</li>
|
||||
<li>often, in addition to sparing traffic on requests, the same concept is applied to responses as well, i.e., no data is returned for modifying operations; thus two clients making simultaneous edits do not see one another's changes.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
@ -2467,7 +2467,7 @@ DELETE /v1/orders/{id}/items/{item_id}
|
||||
<p>Now to reset <code>volume</code> to its default value it's enough to omit it in the <code>PUT /items/{item_id}</code> request body. Also, the operations of deleting one item while simultaneously modifying another one are now transitive.</p>
|
||||
<p>This approach also allows for separating non-mutable and calculated fields (in our case, <code>created_at</code> and <code>status</code>) from editable ones without creating ambiguous situations (what should happen if a client tries to change the <code>created_at</code> field?)</p>
|
||||
<p>It is also possible to return full order objects from <code>PUT</code> endpoints instead of just the sub-resource that was overwritten (though it requires some naming convention).</p>
|
||||
<p><strong>NB</strong>: while decomposing endpoints, the idea of splitting them into mutable and non-mutable data often looks tempting. It makes it possible to mark the latter as infinitely cacheable and never bother about pagination ordering and update format consistency. The plan looks solid on paper, but with the API expansion, it frequently happens that immutable fields eventually cease being immutable, and the entire concept not only stops working properly but even starts looking like a design flaw. We would rather recommend designating data as immutable in one of the two cases: (1) making them editable will really mean breaking backwards compatibility, or (2) the link to the resource (for example, an image) is served via the API as well, and you do possess the capability of making those links persistent (e.g. you might generate a new link to the image instead of rewriting the contents of the old one).</p>
|
||||
<p><strong>NB</strong>: while decomposing endpoints, the idea of splitting them into mutable and non-mutable data often looks tempting. It makes it possible to mark the latter as infinitely cacheable and never bother about pagination ordering and update format consistency. The plan looks solid on paper, but with the API expansion, it frequently happens that immutable fields eventually cease being immutable, and the entire concept not only stops working properly but even starts looking like a design flaw. We would rather recommend designating data as immutable in one of the two cases: (1) making them editable will really mean breaking backwards compatibility, or (2) the link to the resource (for example, an image) is served via the API as well, and you do possess the capability of making those links persistent (e.g., you might generate a new link to the image instead of rewriting the contents of the old one).</p>
|
||||
<p><strong>Even better</strong>: design a format for atomic changes.</p>
|
||||
<pre><code>POST /v1/order/changes
|
||||
X-Idempotency-Token: <idempotency token>
|
||||
@ -2488,8 +2488,8 @@ X-Idempotency-Token: <idempotency token>
|
||||
<h4>Ensuring API Product Quality</h4>
|
||||
<p>Apart from the technological limitations, any real API will soon face the imperfection of the surrounding reality. Of course, any one of us would prefer living in the world of pink unicorns, free of piles of legacy code, evil-doers, national conflicts, and competitors' scheming. Fortunately or not, we live in the real world, and API vendors have to mind all of those while developing the API.</p>
|
||||
<h5><a href="#chapter-11-paragraph-22" id="chapter-11-paragraph-22" class="anchor">22. Use Globally Unique Identifiers</a></h5>
|
||||
<p>It's considered a good form to use globally unique strings as entity identifiers, either semantic (i.e. "lungo" for beverage types) or random ones (i.e. <a href="https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)">UUID-4</a>). It might turn out to be extremely useful if you need to merge data from several sources under a single identifier.</p>
|
||||
<p>In general, we tend to advise using urn-like identifiers, e.g. <code>urn:order:<uuid></code> (or just <code>order:<uuid></code>). That helps a lot in dealing with legacy systems with different identifiers attached to the same entity. Namespaces in urns help to understand quickly which identifier is used and if there is a usage mistake.</p>
|
||||
<p>It's considered a good form to use globally unique strings as entity identifiers, either semantic (e.g., "lungo" for beverage types) or random ones (e.g., <a href="https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)">UUID-4</a>). It might turn out to be extremely useful if you need to merge data from several sources under a single identifier.</p>
|
||||
<p>In general, we tend to advise using urn-like identifiers, e.g., <code>urn:order:<uuid></code> (or just <code>order:<uuid></code>). That helps a lot in dealing with legacy systems with different identifiers attached to the same entity. Namespaces in urns help to understand quickly which identifier is used and if there is a usage mistake.</p>
|
||||
<p>One important implication: <strong>never use increasing numbers as external identifiers</strong>. Apart from the abovementioned reasons, it allows counting how many entities of each type there are in the system. Your competitors will be able to calculate a precise number of orders you have each day, for example.</p>
|
||||
<p><strong>NB</strong>: in this book, we often use short identifiers like "123" in code examples; that's for the convenience of reading the book on small screens. Do not replicate this practice in a real-world API.</p>
|
||||
<h5><a href="#chapter-11-paragraph-23" id="chapter-11-paragraph-23" class="anchor">23. Stipulate Future Restrictions</a></h5>
|
||||
@ -2498,7 +2498,7 @@ X-Idempotency-Token: <idempotency token>
|
||||
<p>It is extremely important to leave room for multi-factored authentication (such as TOTP, SMS, or 3D-secure-like technologies) if it's possible to make payments through the API. In this case, it's a must-have from the very beginning.</p>
|
||||
<h5><a href="#chapter-11-paragraph-24" id="chapter-11-paragraph-24" class="anchor">24. No Bulk Access to Sensitive Data</a></h5>
|
||||
<p>If it's possible to get through the API users' personal data, bank card numbers, private messages, or any other kind of information, exposing which might seriously harm users, partners, and/or you — there must be <em>no</em> methods of bulk getting the data, or at least there must be rate limiters, page size restrictions, and, ideally, multi-factored authentication in front of them.</p>
|
||||
<p>Often, making such offloads on an ad-hoc basis, e.g. bypassing the API, is a reasonable practice.</p>
|
||||
<p>Often, making such offloads on an ad-hoc basis, i.e., bypassing the API, is a reasonable practice.</p>
|
||||
<h5><a href="#chapter-11-paragraph-25" id="chapter-11-paragraph-25" class="anchor">25. Localization and Internationalization</a></h5>
|
||||
<p>All endpoints must accept language parameters (for example, in a form of the <code>Accept-Language</code> header), even if they are not being used currently.</p>
|
||||
<p>It is important to understand that the user's language and the user's jurisdiction are different things. Your API working cycle must always store the user's location. It might be stated either explicitly (requests contain geographical coordinates) or implicitly (initial location-bound request initiates session creation which stores the location), but no correct localization is possible in absence of location data. In most cases reducing the location to just a country code is enough.</p>
|
||||
@ -2674,11 +2674,11 @@ POST /v1/runtimes/{id}/terminate
|
||||
<ol>
|
||||
<li>
|
||||
<p>What does “functionally correctly” mean?</p>
|
||||
<p>It means that the code continues to serve its function, e.g. to solve some users' problems. It doesn't mean it continues working indistinguishably from the previous version: for example, if you're maintaining a UI library, changing functionally insignificant design details like shadow depth or border stroke type is backwards compatible, whereas changing the sizes of the visual components is not.</p>
|
||||
<p>It means that the code continues to serve its function, i.e., to solve some users' problems. It doesn't mean it continues working indistinguishably from the previous version: for example, if you're maintaining a UI library, changing functionally insignificant design details like shadow depth or border stroke type is backwards compatible, whereas changing the sizes of the visual components is not.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>What does “a long period of time” mean?</p>
|
||||
<p>From our point of view, the backwards compatibility maintenance period should be reconciled with the typical lifetime of applications in the subject area. Platform LTS periods are decent guidance in most cases. Since the applications will be rewritten anyway when the platform maintenance period ends, it is reasonable to expect developers to move to the new API version as well. In mainstream subject areas (e.g. desktop and mobile operating systems) this period lasts several years.</p>
|
||||
<p>From our point of view, the backwards compatibility maintenance period should be reconciled with the typical lifetime of applications in the subject area. Platform LTS periods are decent guidance in most cases. Since the applications will be rewritten anyway when the platform maintenance period ends, it is reasonable to expect developers to move to the new API version as well. In mainstream subject areas (i.e., desktop and mobile operating systems) this period lasts several years.</p>
|
||||
</li>
|
||||
</ol>
|
||||
<p>From the definition becomes obvious why backwards compatibility needs to be maintained (including taking necessary measures at the API design stage). An outage, full or partial, caused by an API vendor, is an extremely uncomfortable situation for every developer, if not a disaster — especially if they pay money for the API usage.</p>
|
||||
@ -2709,7 +2709,7 @@ POST /v1/runtimes/{id}/terminate
|
||||
<li>
|
||||
<p>If the code-on-demand feature isn't supported or is prohibited by the platform, as in modern mobile operating systems, then the situation becomes more severe. Each client effectively borrows a snapshot of the code working with your API, frozen at the moment of compilation. Client application updates are scattered over time to much more extent than Web application updates. The most painful thing is that <em>some clients will never be up to date</em>, because one of three reasons:</p>
|
||||
<ul>
|
||||
<li>developers simply don't want to update the app, e.g. its development stopped;</li>
|
||||
<li>developers simply don't want to update the app, i.e., its development stopped;</li>
|
||||
<li>users don't want to get updates (sometimes because users think that developers “spoiled” the app in new versions);</li>
|
||||
<li>users can't get updates because their devices are no longer supported.</li>
|
||||
</ul>
|
||||
@ -2726,7 +2726,7 @@ POST /v1/runtimes/{id}/terminate
|
||||
<li>interfaces change.</li>
|
||||
</ul>
|
||||
<p>As usual, the API provides an abstraction to a much more granular subject area. In the case of our coffee machine API example, one might reasonably expect new machine models to pop up, which are to be supported by the platform. New models tend to provide new APIs, and it's hard to guarantee they might be adopted while preserving the same high-level API. And anyway, the code needs to be altered, which might lead to incompatibility, albeit unintentional.</p>
|
||||
<p>Let us also stress that vendors of low-level API are not always as resolute regarding maintaining backwards compatibility for their APIs (actually, any software they provide) as (we hope so) you are. You should be warned that keeping your API in an operational state, e.g. writing and supporting facades to the shifting subject area landscape, will be a problem of yours, and sometimes rather a sudden one.</p>
|
||||
<p>Let us also stress that vendors of low-level API are not always as resolute regarding maintaining backwards compatibility for their APIs (actually, any software they provide) as (we hope so) you are. You should be warned that keeping your API in an operational state, i.e., writing and supporting facades to the shifting subject area landscape, will be a problem of yours, and sometimes rather a sudden one.</p>
|
||||
<h4>Platform Drift</h4>
|
||||
<p>Finally, there is a third side to the story — the “canyon” you're crossing over with a bridge of your API. Developers write code that is executed in some environment you can't control, and it's evolving. New versions of operating systems, browsers, protocols, and programming language SDKs emerge. New standards are being developed and new arrangements made, some of them being backwards-incompatible, and nothing could be done about that.</p>
|
||||
<p>Older platform versions lead to fragmentation just like older app versions do, because developers (including the API developers) are struggling with supporting older platforms, and users are struggling with platform updates — and often can't get updated at all, since newer platform versions require newer devices.</p>
|
||||
@ -2771,7 +2771,7 @@ POST /v1/runtimes/{id}/terminate
|
||||
<p><strong>NB</strong>. Sometimes to defend the single accessible API version concept, the following argument is put forward: preserving the SDK or API application server code is not enough to maintain strict backwards compatibility as it might be relying on some un-versioned services (for example, some data in the DB that are shared between all the API versions). We, however, consider this an additional reason to isolate such dependencies (see <a href="#back-compat-serenity-notepad">“The Serenity Notepad”</a> chapter) as it means that changes to these subsystems might lead to the inoperability of the API.</p><div class="page-break"></div><h3><a href="#back-compat-iceberg-waterline" class="anchor" id="back-compat-iceberg-waterline">Chapter 14. On the Waterline of the Iceberg</a><a href="#chapter-14" class="secondary-anchor" id="chapter-14"> </a></h3>
|
||||
<p>Before we start talking about the extensible API design, we should discuss the hygienic minimum. A huge number of problems would have never happened if API vendors had paid more attention to marking their area of responsibility.</p>
|
||||
<h4>Provide a Minimal Amount of Functionality</h4>
|
||||
<p>At any moment in its lifetime, your API is like an iceberg: it comprises an observable (e.g. documented) part and a hidden one, undocumented. If the API is designed properly, these two parts correspond to each other just like the above-water and under-water parts of a real iceberg do, i.e. one to ten. Why so? Because of two obvious reasons.</p>
|
||||
<p>At any moment in its lifetime, your API is like an iceberg: it comprises an observable (i.e., documented) part and a hidden one, undocumented. If the API is designed properly, these two parts correspond to each other just like the above-water and under-water parts of a real iceberg do, i.e. one to ten. Why so? Because of two obvious reasons.</p>
|
||||
<ul>
|
||||
<li>
|
||||
<p>Computers exist to make complicated things easy, not vice versa. The code developers write upon your API must describe a complicated problem's solution in neat and straightforward sentences. If developers have to write more code than the API itself comprises, then there is something rotten here. Probably, this API simply isn't needed at all.</p>
|
||||
@ -3114,7 +3114,7 @@ PUT /formatters/volume/ru/US
|
||||
…
|
||||
}
|
||||
</code></pre>
|
||||
<p>We should also note that providing a newly created entity identifier by the requesting side isn't exactly the best practice. However, since we decided from the very beginning to keep recipe identifiers semantically meaningful, we have to live on with this convention. Obviously, we're risking getting lots of collisions on recipe names used by different partners, so we actually need to modify this operation: either a partner must always use a pair of identifiers (i.e. the recipe id plus the partner's own id), or we need to introduce composite identifiers, as we recommended earlier in the <a href="#api-design-describing-interfaces">“Describing Final Interfaces”</a> chapter.</p>
|
||||
<p>We should also note that providing a newly created entity identifier by the requesting side isn't exactly the best practice. However, since we decided from the very beginning to keep recipe identifiers semantically meaningful, we have to live on with this convention. Obviously, we're risking getting lots of collisions on recipe names used by different partners, so we actually need to modify this operation: either a partner must always use a pair of identifiers (e.g., the recipe id plus the partner's own id), or we need to introduce composite identifiers, as we recommended earlier in the <a href="#api-design-describing-interfaces">“Describing Final Interfaces”</a> chapter.</p>
|
||||
<pre><code>POST /v1/recipes/custom
|
||||
{
|
||||
// The first part of the composite
|
||||
@ -3171,7 +3171,7 @@ PUT /formatters/volume/ru/US
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
<p><strong>NB</strong>: by doing so, we transfer the complexity of developing the API onto the plane of developing appropriate data formats, e.g. developing formats for order parameters to the <code>program_run_endpoint</code>, and what format the <code>program_get_state_endpoint</code> shall return, etc., but in this chapter, we're focusing on different questions.</p>
|
||||
<p><strong>NB</strong>: by doing so, we transfer the complexity of developing the API onto the plane of developing appropriate data formats, i.e., developing formats for order parameters to the <code>program_run_endpoint</code>, and what format the <code>program_get_state_endpoint</code> shall return, etc., but in this chapter, we're focusing on different questions.</p>
|
||||
<p>Though this API looks absolutely universal, it's quite easy to demonstrate how once simple and clear API ends up being confusing and convoluted. This design presents two main problems:</p>
|
||||
<ol>
|
||||
<li>It describes nicely the integrations we've already implemented (it costs almost nothing to support the API types we already know) but brings no flexibility to the approach. In fact, we simply described what we'd already learned, not even trying to look at the larger picture.</li>
|
||||
@ -3186,7 +3186,7 @@ PUT /formatters/volume/ru/US
|
||||
</ul>
|
||||
<p>Furthermore, we have to describe both endpoints in the docs. It's quite natural that the <code>takeout</code> endpoint is very specific; unlike requesting contactless delivery, which we hid under the pretty general <code>modify</code> endpoint, operations like takeout approval will require introducing a new unique method every time. After several iterations, we would have a scrapyard, full of similarly looking methods, mostly optional — but developers would need to study the docs nonetheless to understand, which methods are needed in your specific situation, and which are not.</p>
|
||||
<p><strong>NB</strong>: in this example, we assumed that passing <code>program_takeout_endpoint</code> parameter is the flag to the application to display the “get the order” button; it would be better to add something like a <code>supported_flow</code> field to the <code>PUT /api-types/</code> endpoint to provide an explicit flag instead of this implicit convention; however, this wouldn't change the problematics of stockpiling optional methods in the interface, so we skipped it to keep examples laconic.</p>
|
||||
<p>We actually don't know, whether in the real world of coffee machine APIs this problem will occur or not. But we can say with all confidence regarding “bare metal” integrations that the processes we described <em>always</em> happen. The underlying technology shifts; an API that seemed clear and straightforward, becomes a trash bin full of legacy methods, half of which borrows no practical sense under any specific set of conditions. If we add technical progress to the situation, i.e. imagine that after a while all coffee houses have become automated, we will finally end up with the situation with half of the methods <em>aren't actually needed at all</em>, like requesting a contactless takeout one.</p>
|
||||
<p>We actually don't know, whether in the real world of coffee machine APIs this problem will occur or not. But we can say with all confidence regarding “bare metal” integrations that the processes we described <em>always</em> happen. The underlying technology shifts; an API that seemed clear and straightforward, becomes a trash bin full of legacy methods, half of which borrows no practical sense under any specific set of conditions. If we add technical progress to the situation, e.g., imagine that after a while all coffee houses have become automated, we will finally end up with the situation with half of the methods <em>aren't actually needed at all</em>, like requesting a contactless takeout one.</p>
|
||||
<p>It is also worth mentioning that we unwittingly violated the abstraction levels isolation principle. At the vending machine API level, there is no such thing as a “contactless takeout,” that's actually a product concept.</p>
|
||||
<p>So, how would we tackle this issue? Using one of two possible approaches: either thoroughly study the entire subject area and its upcoming improvements for at least several years ahead, or abandon strong coupling in favor of a weak one. How would the <em>ideal</em> solution look to both parties? Something like this:</p>
|
||||
<ul>
|
||||
@ -3247,7 +3247,7 @@ registerProgramRunHandler(
|
||||
<li>a low-level context doesn't know anything about alternative implementations — and it really doesn't; it handles only those events which mean something at its level and emits only those events that could happen under its specific conditions.</li>
|
||||
</ul>
|
||||
<p>It's ultimately possible that both sides would know nothing about each other and wouldn't interact at all, and this might happen with the evolution of underlying technologies.</p>
|
||||
<p><strong>NB</strong>: in the real world this might not be the case, e.g. we might <em>want</em> the application to know, whether the takeout request was successfully served or not, e.g. listen to the <code>takeout_ready</code> event and require the <code>takeout_ready</code> flag in the state of the execution context. Still, the general possibility of <em>not caring</em> about the implementation details is a very powerful technique that makes the application code much less complex — of course, unless this knowledge is important to the user.</p>
|
||||
<p><strong>NB</strong>: in the real world this might not be the case, i.e., we might <em>want</em> the application to know, whether the takeout request was successfully served or not, i.e., listen to the <code>takeout_ready</code> event and require the <code>takeout_ready</code> flag in the state of the execution context. Still, the general possibility of <em>not caring</em> about the implementation details is a very powerful technique that makes the application code much less complex — of course, unless this knowledge is important to the user.</p>
|
||||
<p>One more important feature of weak coupling is that it allows an entity to have several higher-level contexts. In typical subject areas, such a situation would look like an API design flaw, but in complex systems, with several system state-modifying agents present, such design patterns are not that rare. Specifically, you would likely face it while developing user-facing UI libraries. We will cover this issue in detail in the “SDK and UI Libraries” section of this book.</p>
|
||||
<h4>The Inversion of Responsibility</h4>
|
||||
<p>It becomes obvious from what was said above that two-way weak coupling means a significant increase in code complexity on both levels, which is often redundant. In many cases, two-way event linking might be replaced with one-way linking without significant loss of design quality. That means allowing a low-level entity to call higher-level methods directly instead of generating events. Let's alter our example:</p>
|
||||
@ -3289,7 +3289,7 @@ registerProgramRunHandler(
|
||||
<p>Another reason to justify this solution is that major changes occurring at different abstraction levels have different weights:</p>
|
||||
<ul>
|
||||
<li>if the technical level is under change, that must not affect product qualities and the code written by partners;</li>
|
||||
<li>if the product is changing, i.e. we start selling flight tickets instead of preparing coffee, there is literally no sense to preserve backwards compatibility at technical abstraction levels. Ironically, we may actually make our API sell tickets instead of brewing coffee without breaking backwards compatibility, but the partners' code will still become obsolete.</li>
|
||||
<li>if the product is changing, e.g., we start selling flight tickets instead of preparing coffee, there is literally no sense to preserve backwards compatibility at technical abstraction levels. Ironically, we may actually make our API sell tickets instead of brewing coffee without breaking backwards compatibility, but the partners' code will still become obsolete.</li>
|
||||
</ul>
|
||||
<p>In conclusion, as higher-level APIs are evolving more slowly and much more consistently than low-level APIs, reverse strong coupling might often be acceptable or even desirable, at least from the price-quality ratio point of view.</p>
|
||||
<p><strong>NB</strong>: many contemporary frameworks explore a shared state approach, Redux being probably the most notable example. In the Redux paradigm, the code above would look like this:</p>
|
||||
@ -3306,7 +3306,7 @@ registerProgramRunHandler(
|
||||
}
|
||||
);
|
||||
</code></pre>
|
||||
<p>Let us note that this approach <em>in general</em> doesn't contradict the weak coupling principle, but violates another one — of abstraction levels isolation, and therefore isn't very well suited for writing branchy APIs with high hierarchy trees. In such systems, it's still possible to use a global or quasi-global state manager, but you need to implement event or method call propagation through the hierarchy, e.g. ensure that a low-level entity always interacts with its closest higher-level neighbors only, delegating the responsibility of calling high-level or global methods to them.</p>
|
||||
<p>Let us note that this approach <em>in general</em> doesn't contradict the weak coupling principle, but violates another one — of abstraction levels isolation, and therefore isn't very well suited for writing branchy APIs with high hierarchy trees. In such systems, it's still possible to use a global or quasi-global state manager, but you need to implement event or method call propagation through the hierarchy, i.e., ensure that a low-level entity always interacts with its closest higher-level neighbors only, delegating the responsibility of calling high-level or global methods to them.</p>
|
||||
<pre><code>program.context.on(
|
||||
'takeout_requested',
|
||||
() => {
|
||||
@ -3333,14 +3333,14 @@ ProgramContext.dispatch = (action) => {
|
||||
}
|
||||
</code></pre>
|
||||
<h4>Delegate!</h4>
|
||||
<p>From what was said, one more important conclusion follows: doing a real job, e.g. implementing some concrete actions (making coffee, in our case) should be delegated to the lower levels of the abstraction hierarchy. If the upper levels try to prescribe some specific implementation algorithms, then (as we have demonstrated on the <code>order_execution_endpoint</code> example) we will soon face a situation of inconsistent methods and interaction protocols nomenclature, most of which have no specific meaning when we talk about some specific hardware context.</p>
|
||||
<p>From what was said, one more important conclusion follows: doing a real job, i.e., implementing some concrete actions (making coffee, in our case) should be delegated to the lower levels of the abstraction hierarchy. If the upper levels try to prescribe some specific implementation algorithms, then (as we have demonstrated on the <code>order_execution_endpoint</code> example) we will soon face a situation of inconsistent methods and interaction protocols nomenclature, most of which have no specific meaning when we talk about some specific hardware context.</p>
|
||||
<p>Contrariwise, applying the paradigm of concretizing the contexts at each new abstraction level, we will eventually fall into the bunny hole deep enough to have nothing to concretize: the context itself unambiguously matches the functionality we can programmatically control. And at that level, we must stop detailing contexts further, and just realize the algorithms needed. It's worth mentioning that the abstraction deepness for different underlying platforms might vary.</p>
|
||||
<p><strong>NB</strong>. In the <a href="#api-design-separating-abstractions">“Separating Abstraction Levels”</a> chapter we have illustrated exactly this: when we speak about the first coffee machine API type, there is no need to extend the tree of abstractions further than running programs, but with the second API type, we need one more intermediary abstraction level, namely the runtimes API.</p><div class="page-break"></div><h3><a href="#back-compat-universal-interfaces" class="anchor" id="back-compat-universal-interfaces">Chapter 18. Interfaces as a Universal Pattern</a><a href="#chapter-18" class="secondary-anchor" id="chapter-18"> </a></h3>
|
||||
<p>Let us summarize what we have written in the three previous chapters:</p>
|
||||
<ol>
|
||||
<li>Extending API functionality is implemented through abstracting: the entity nomenclature is to be reinterpreted so that existing methods become partial (ideally — the most frequent) simplified cases to more general functionality.</li>
|
||||
<li>Higher-level entities are to be the informational contexts for low-level ones, e.g. don't prescribe any specific behavior but translate their state and expose functionality to modify it (directly through calling some methods or indirectly through firing events).</li>
|
||||
<li>Concrete functionality, e.g. working with “bare metal” hardware or underlying platform APIs, should be delegated to low-level entities.</li>
|
||||
<li>Higher-level entities are to be the informational contexts for low-level ones, i.e., don't prescribe any specific behavior but translate their state and expose functionality to modify it (directly through calling some methods or indirectly through firing events).</li>
|
||||
<li>Concrete functionality, i.e., working with “bare metal” hardware or underlying platform APIs, should be delegated to low-level entities.</li>
|
||||
</ol>
|
||||
<p><strong>NB</strong>. There is nothing novel about these rules: one might easily recognize them being the <a href="https://en.wikipedia.org/wiki/SOLID">SOLID</a> architecture principles. There is no surprise in that either, because SOLID concentrates on contract-oriented development, and APIs are contracts by definition. We've just added the “abstraction levels” and “informational contexts” concepts there.</p>
|
||||
<p>However, there is an unanswered question: how should we design the entity nomenclature from the beginning so that extending the API won't make it a mess of different inconsistent methods of different ages? The answer is pretty obvious: to avoid clumsy situations while abstracting (as with the recipe properties), all the entities must be originally considered being a specific implementation of a more general interface, even if there are no planned alternative implementations for them.</p>
|
||||
@ -3387,7 +3387,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>usually, you have no guarantees that the partner will maintain backwards compatibility or at least keep new versions more or less conceptually akin to the older ones;</li>
|
||||
<li>any partner's problem will automatically ricochet into your customers.</li>
|
||||
</ul>
|
||||
<p>The best practice is quite the opposite: isolate the third-party API usage, e.g. develop an abstraction level that will allow for:</p>
|
||||
<p>The best practice is quite the opposite: isolate the third-party API usage, i.e., develop an abstraction level that will allow for:</p>
|
||||
<ul>
|
||||
<li>keeping backwards compatibility intact because of extension capabilities incorporated in the API design;</li>
|
||||
<li>negating partner's problems by technical means:
|
||||
@ -3408,7 +3408,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>There are obvious local problems with this approach (like the inconsistency in functions' behavior, or the bugs which were not found while testing the code), but also a bigger one: your API might be simply unusable if a developer tries any non-mainstream approach, because of performance issues, bugs, instability, etc., as the API developers themselves never tried to use this public interface for anything important.</p>
|
||||
<p><strong>NB</strong>. The perfect example of avoiding this anti-pattern is the development of compilers; usually, the next compiler's version is compiled with the previous compiler's version.</p>
|
||||
<h5><a href="#chapter-19-paragraph-5" id="chapter-19-paragraph-5" class="anchor">5. Keep a Notepad</a></h5>
|
||||
<p>Whatever tips and tricks described in the previous chapters you use, it's often quite probable that you can't do <em>anything</em> to prevent API inconsistencies from piling up. It's possible to reduce the speed of this stockpiling, foresee some problems, and have some interface durability reserved for future use. But one can't foresee <em>everything</em>. At this stage, many developers tend to make some rash decisions, e.g. releasing a backwards-incompatible minor version to fix some design flaws.</p>
|
||||
<p>Whatever tips and tricks described in the previous chapters you use, it's often quite probable that you can't do <em>anything</em> to prevent API inconsistencies from piling up. It's possible to reduce the speed of this stockpiling, foresee some problems, and have some interface durability reserved for future use. But one can't foresee <em>everything</em>. At this stage, many developers tend to make some rash decisions, e.g., releasing a backwards-incompatible minor version to fix some design flaws.</p>
|
||||
<p>We highly recommend never doing that. Remember that the API is also a multiplier of your mistakes. What we recommend is to keep a serenity notepad — to write down the lessons learned, and not to forget to apply this knowledge when a new major API version is released.</p><div class="page-break"></div><h2><a href="#section-4" class="anchor" id="section-4">Section III. The API Product</a></h2><h3><a href="#api-product" class="anchor" id="api-product">Chapter 20. API as a Product</a><a href="#chapter-20" class="secondary-anchor" id="chapter-20"> </a></h3>
|
||||
<p>There are two important statements regarding APIs viewed as products.</p>
|
||||
<ol>
|
||||
@ -3434,7 +3434,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>Finally, customers don't care about the technical aspects of the solution; they choose the product they need and ask for some specific functionality implemented.</li>
|
||||
</ul>
|
||||
<p>So it turns out that customers are at the apex of the pyramid: it is customers you need to convince they need not <em>any</em> cup of coffee, but a cup of coffee brewed using our API (interesting question: how will we convey the knowledge which API works under the hood, and why customers should pay their money for it!); then business owners will set the task to integrate the API, and developers will have no other choice but to implement it (which, by the way, means that investing into API's readability and consistency is not <em>that</em> important).</p>
|
||||
<p>The truth, of course, lies somewhere in between. In some markets and subject areas, it is developers who make decisions (i.e. which framework to choose); in other markets and areas, it might be business owners or customers. It also depends on the competitiveness of the market: introducing a new frontend framework does not meet any resistance while developing, let's say, a new mobile operating system requires million-dollar investments into promotions and strategic partnerships.</p>
|
||||
<p>The truth, of course, lies somewhere in between. In some markets and subject areas, it is developers who make decisions (e.g., which framework to choose); in other markets and areas, it might be business owners or customers. It also depends on the competitiveness of the market: introducing a new frontend framework does not meet any resistance while developing, let's say, a new mobile operating system requires million-dollar investments into promotions and strategic partnerships.</p>
|
||||
<p>Here and after, we will describe some “averaged” situations, meaning that all three pyramid levels are important: customers choosing the product which fits their needs best, business owners seeking quality guarantees and lower development costs, as well as software engineers caring about the API capabilities and the convenience of working with it.</p><div class="page-break"></div><h3><a href="#api-product-business-models" class="anchor" id="api-product-business-models">Chapter 21. The API Business Models</a><a href="#chapter-21" class="secondary-anchor" id="chapter-21"> </a></h3>
|
||||
<p>Before we proceed to the API product management principles, let us draw your attention to the matter of profits that the API vendor company might extract from it. As we will demonstrate in the next chapters, this is not an idle question as it directly affects making product decisions and setting KPIs for the API team. In this chapter, we will enumerate the main API monetization models. [In brackets, we will provide examples of such models applicable to our coffee-machine API study.]</p>
|
||||
<h5><a href="#chapter-21-paragraph-1" id="chapter-21-paragraph-1" class="anchor">1. Developers = End Users</a></h5>
|
||||
@ -3442,10 +3442,10 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>There is also a plethora of monetizing techniques; in fact, we're just talking about monetizing software for developers.</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>The framework / library / platform might be paid per se, e.g. distributed under a commercial license. Nowadays such models are becoming less and less popular with the rise of free and open-source software but are still quite common.</p>
|
||||
<p>The framework / library / platform might be paid per se, i.e., distributed under a commercial license. Nowadays such models are becoming less and less popular with the rise of free and open-source software but are still quite common.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>The API may be licensed under an open license with some restrictions that might be lifted by buying an extended license. It might be either functional limitations (an inability to publish the app in the app store or an incapacity to build the app in the production mode) or usage restrictions (for example, using the API for some purposes might be prohibited or an open license might be “contagious,” e.g. require publishing the derived code under the same license).</p>
|
||||
<p>The API may be licensed under an open license with some restrictions that might be lifted by buying an extended license. It might be either functional limitations (an inability to publish the app in the app store or an incapacity to build the app in the production mode) or usage restrictions (for example, using the API for some purposes might be prohibited or an open license might be “contagious,” i.e., require publishing the derived code under the same license).</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>The API itself might be free, but the API vendor might provide additional paid services (for example, consulting or integrating ones), or just sell the extended technical support.</p>
|
||||
@ -3496,15 +3496,15 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>you might concentrate efforts on increasing awareness among business owners [for example, for partners integrating a coffee ordering widget on their websites to also pay attention to your tires catalog];</li>
|
||||
<li>finally, you might provide APIs only to make developers know your company's name to increase their knowledge of your other products or just to improve your reputation as an employer (this activity is sometimes called “tech-PR”).</li>
|
||||
</ul>
|
||||
<p>Additionally, we might talk about forming a community, e.g. a network of developers (or customers, or business owners) who are loyal to the product. The benefits of having such a community might be substantial: lowering the technical support costs, getting a convenient channel for publishing announcements regarding new services and new releases, and obtaining beta users for upcoming products.</p>
|
||||
<p>Additionally, we might talk about forming a community, i.e., a network of developers (or customers, or business owners) who are loyal to the product. The benefits of having such a community might be substantial: lowering the technical support costs, getting a convenient channel for publishing announcements regarding new services and new releases, and obtaining beta users for upcoming products.</p>
|
||||
<h5><a href="#chapter-21-paragraph-7" id="chapter-21-paragraph-7" class="anchor">7. API = a Feedback and UGC Tool</a></h5>
|
||||
<p>If a company possesses some big data, it might be useful to provide a public API for users to make corrections in the data or otherwise get involved in working with it. For example, cartographical API providers usually allow to post feedback or correct a mistake right on partners' websites and applications. [In the case of our coffee API, we might be collecting feedback to improve the service, both passively through building coffeeshops ratings or actively through contacting business owners to convey users' requests or through finding new coffee shops that are still not integrated with the platform.]</p>
|
||||
<h5><a href="#chapter-21-paragraph-8" id="chapter-21-paragraph-8" class="anchor">8. Terraforming</a></h5>
|
||||
<p>Finally, the most altruistic approach to API product development is providing it free of charge (or as an open source and open data project) just to change the landscape. If today nobody's willing to pay for the API, we might invest in popularizing the functionality hoping to find commercial niches later (in any of the abovementioned formats) or to increase the significance and usefulness of the API integrations for end users (and therefore the readiness of the partners to pay for the API). [In the case of our coffee example, imagine a coffee machine maker that starts providing APIs for free aiming to make having an API a “must” for every coffee machine vendor thus allowing for the development of commercial API-based services in the future.]</p>
|
||||
<h5><a href="#chapter-21-paragraph-9" id="chapter-21-paragraph-9" class="anchor">9. Gray Zones</a></h5>
|
||||
<p>One additional source of income for the API provider is the analysis of the requests that end users make. In other words — collecting and re-selling some user data. You must be aware that the difference between acceptable data collecting (such as aggregating search requests to understand trends or finding promising locations for opening a coffee shop) and unacceptable ones are quite vague, and tends to vary in time and space (e.g. some actions might be totally legal at the one side of the state border, and totally illegal at the other side), so making a decision of monetizing the API with it should be carried out with extreme caution.</p>
|
||||
<p>One additional source of income for the API provider is the analysis of the requests that end users make. In other words — collecting and re-selling some user data. You must be aware that the difference between acceptable data collecting (such as aggregating search requests to understand trends or finding promising locations for opening a coffee shop) and unacceptable ones are quite vague, and tends to vary in time and space (e.g., some actions might be totally legal at the one side of the state border, and totally illegal at the other side), so making a decision of monetizing the API with it should be carried out with extreme caution.</p>
|
||||
<h4>The API-First Approach</h4>
|
||||
<p>Last several years we see the trend of providing some functionality as an API (e.g. as a product for developers) instead of developing the service for end users. This approach, dubbed “API-first,” reflects the growing specialization in the IT world: developing APIs becomes a separate area of expertise that businesses are ready to outsource instead of spending resources to develop internal APIs for their applications by the in-house IT department. However, this approach is not universally accepted (yet), and you should keep in mind the factors that affect the decision of launching a service in the API-first paradigm:</p>
|
||||
<p>Last several years we see the trend of providing some functionality as an API (i.e., as a product for developers) instead of developing the service for end users. This approach, dubbed “API-first,” reflects the growing specialization in the IT world: developing APIs becomes a separate area of expertise that businesses are ready to outsource instead of spending resources to develop internal APIs for their applications by the in-house IT department. However, this approach is not universally accepted (yet), and you should keep in mind the factors that affect the decision of launching a service in the API-first paradigm:</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>The target market must be sufficiently heated up: there must be companies there that possess enough resources to develop services atop third-party APIs and pay for it (unless your aim is terraforming).</p>
|
||||
@ -3516,8 +3516,8 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>You must really possess expertise in API development; otherwise, there are high chances to make too many design mistakes.</p>
|
||||
</li>
|
||||
</ol>
|
||||
<p>Sometimes providing APIs is a method to “probe the ground,” e.g. to evaluate the market and decide whether it's worth having a full-scale user service there. (We rather condemn this practice as it inevitably leads to discontinuing the API or limiting its functionality, either because the market turns out to be not as profitable as expected, or because the API eventually becomes a competitor to the main service.)</p><div class="page-break"></div><h3><a href="#api-product-vision" class="anchor" id="api-product-vision">Chapter 22. Developing a Product Vision</a><a href="#chapter-22" class="secondary-anchor" id="chapter-22"> </a></h3>
|
||||
<p>The above-mentioned fragmentation of the API target audience, e.g. the “developers — business — end users” triad, makes API product management quite a non-trivial problem. Yes, the basics are the same: find your auditory's needs and satisfy them; the problem is that your product has several different audiences, and their interests sometimes diverge. The end users' request for an affordable cup of coffee does not automatically imply business demand for a coffee machine API.</p>
|
||||
<p>Sometimes providing APIs is a method to “probe the ground,” i.e., to evaluate the market and decide whether it's worth having a full-scale user service there. (We rather condemn this practice as it inevitably leads to discontinuing the API or limiting its functionality, either because the market turns out to be not as profitable as expected, or because the API eventually becomes a competitor to the main service.)</p><div class="page-break"></div><h3><a href="#api-product-vision" class="anchor" id="api-product-vision">Chapter 22. Developing a Product Vision</a><a href="#chapter-22" class="secondary-anchor" id="chapter-22"> </a></h3>
|
||||
<p>The above-mentioned fragmentation of the API target audience, i.e., the “developers — business — end users” triad, makes API product management quite a non-trivial problem. Yes, the basics are the same: find your auditory's needs and satisfy them; the problem is that your product has several different audiences, and their interests sometimes diverge. The end users' request for an affordable cup of coffee does not automatically imply business demand for a coffee machine API.</p>
|
||||
<p>Generally speaking, the API product vision must include the same three elements:</p>
|
||||
<ul>
|
||||
<li>grasping how <em>end users</em> would like to have their problems solved;</li>
|
||||
@ -3534,10 +3534,10 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>The same fuzziness should be kept in mind while making interviews and getting feedback. Software engineers will mainly report the problems they've got with the technical integrations, and rarely speak of business-related issues; meanwhile, business owners care little about the inconvenience of writing code. Both will have some knowledge regarding the end users' problems, but it's usually limited to the market segment the partner operates on.</p>
|
||||
<p>If you do have an access to end users' actions monitoring (see <a href="#api-product-kpi">“The API Key Performance Indicators”</a> chapter), then you might try to analyze the typical user behavior through these logs and understand how users interact with the partners' applications. But you will need to make this analysis on a per-application basis and try to clusterize the most common scenarios.</p>
|
||||
<h4>Checking Product Hypotheses</h4>
|
||||
<p>Apart from the general complexity of formulating the product vision, there are also tactical issues with checking product hypotheses. “The Holy Grail” of product management — that is, creating a cheap (in terms of resource spent) minimal viable product (MVP) — is normally unavailable for an API product manager. The thing is that you can't easily <em>test</em> the solution even if you managed to develop an API MVP: to do so, partners are to <em>develop some code</em>, e.g. invest their money; and if the outcome of the experiment is negative (e.g. the further development looks unpromising), this money will be wasted. Of course, partners will be a little bit skeptical towards such proposals. Thus a “cheap” MVP should include either the compensation for partners' expenses or the budget to develop a reference implementation (e.g. a complementary application that is created to support the API MVP).</p>
|
||||
<p>Apart from the general complexity of formulating the product vision, there are also tactical issues with checking product hypotheses. “The Holy Grail” of product management — that is, creating a cheap (in terms of resource spent) minimal viable product (MVP) — is normally unavailable for an API product manager. The thing is that you can't easily <em>test</em> the solution even if you managed to develop an API MVP: to do so, partners are to <em>develop some code</em>, i.e., invest their money; and if the outcome of the experiment is negative (i.e., the further development looks unpromising), this money will be wasted. Of course, partners will be a little bit skeptical towards such proposals. Thus a “cheap” MVP should include either the compensation for partners' expenses or the budget to develop a reference implementation (i.e., a complementary application that is created to support the API MVP).</p>
|
||||
<p>You might partially solve the problem by making some third-party company release the MVP (for example, in a form of an open-source module published in some developer's personal repository) but then you will struggle with hypothesis validation issues as such modules might easily go unnoticed.</p>
|
||||
<p>Another option for checking conjectures is recruiting some other developers within the API provider company to try the API in their services. Internal customers are usually much more loyal towards spending some effort to check a hypothesis, and it's much easier to negotiate MVP curtailing or freezing with them. The problem is that you can check only those ideas that are relevant to the company's internal needs.</p>
|
||||
<p>Also, applying the “eat your own dog food” concept to APIs means that the API product team should have their own test applications (i.e. “pet projects”) on top of the API. Given the complexity of developing such applications, it makes sense to encourage having them, e.g. giving free API quotas to team members and providing sufficient free computational resources.</p>
|
||||
<p>Also, applying the “eat your own dog food” concept to APIs means that the API product team should have their own test applications (so-called “pet projects”) on top of the API. Given the complexity of developing such applications, it makes sense to encourage having them, i.e., giving free API quotas to team members and providing sufficient free computational resources.</p>
|
||||
<p>Such pet projects are also valuable because of the unique experience they allow to gain: everyone might try a new role. Developers will learn product managers' typical problems: it's not enough to write fine code, you also need to know your customer, understand their demands, formulate an attractive concept, and communicate it. In their turn, product managers will get some understanding of how exactly easy or hard it is to render their product vision into life, and what problems the implementation will bring. Finally, both will benefit from taking a fresh look at the API documentation and putting themselves in the shoes of a developer who heard about the API product for the first time and is now struggling with grasping the basics.</p><div class="page-break"></div><h3><a href="#api-product-devrel" class="anchor" id="api-product-devrel">Chapter 23. Communicating with Developers</a><a href="#chapter-23" class="secondary-anchor" id="chapter-23"> </a></h3>
|
||||
<p>As we have described in the previous chapters, managing an API product requires building relations with both business partners and developers. (Ideally, with end users as well; though this option is seldom available to API providers.)</p>
|
||||
<p>Let's start with developers. The specifics of software engineers as an auditory are the following:</p>
|
||||
@ -3621,9 +3621,9 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>The important rule of API product management any major API provider will soon learn is: don't just ship one specific API; there is always room for a range of products, and this range is two-dimensional.</p>
|
||||
<h4>Horizontal Scaling of API Services</h4>
|
||||
<p>Usually, any functionality available through an API might be split into independent units. For example, in our coffee API, there are offer search endpoints and order processing endpoints. Nothing could prevent us from either pronouncing those functional clusters different APIs or, vice versa, considering them as parts of one API.</p>
|
||||
<p>Different companies employ different approaches to determining the granularity of API services, e.g. what is counted as a separate product and what is not. To some extent, this is a matter of convenience and taste judgment. Consider splitting an API into parts if:</p>
|
||||
<p>Different companies employ different approaches to determining the granularity of API services, i.e., what is counted as a separate product and what is not. To some extent, this is a matter of convenience and taste judgment. Consider splitting an API into parts if:</p>
|
||||
<ul>
|
||||
<li>it makes sense for partners to integrate only one API part, e.g. some isolated subset of the API provides enough means to solve users' problems;</li>
|
||||
<li>it makes sense for partners to integrate only one API part, i.e., some isolated subset of the API provides enough means to solve users' problems;</li>
|
||||
<li>API parts might be versioned separately and independently, and it is meaningful from the partners' point of view (this usually means that those “isolated” APIs are either fully independent or maintain strict backwards compatibility and introduce new major versions only when it's absolutely necessary; otherwise, maintaining a matrix which API #1 version is compatible with which API #2 version will soon become a catastrophe);</li>
|
||||
<li>it makes sense to set tariffs and limits for each API service independently;</li>
|
||||
<li>the auditory of the API segments (either developers, business owners, or end users) is not overlapping, and “selling” granular API to customers is much easier than aggregated.</li>
|
||||
@ -3638,7 +3638,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>The next simplification step is providing services for code generation. In this service, developers choose one of the pre-built integration templates, customize some options, and got a ready-to-use piece of code that might be simply copy-pasted into the application code (and might be additionally customized by adding some level 1-3 code). This approach is sometimes called “point-and-click programming.” [In the case of our coffee API, an example of such a service might have a form or screen editor for a developer to place UI elements and get the working application code.]</li>
|
||||
<li>Finally, this approach might be simplified even further if the service generates not code but a ready-to-use component / widget / frame and a one-liner to integrate it. [For example, if we allow embedding an iframe that handles the entire coffee ordering process right on the partner's website, or describes the rules of forming the image URL that will show the most relevant offer to an end user if embedded in the partner's app.]</li>
|
||||
</ol>
|
||||
<p>Ultimately, we will end up with a concept of meta-API, e.g. those high-level components will have an API of their own built on top of the basic API.</p>
|
||||
<p>Ultimately, we will end up with a concept of meta-API, i.e., those high-level components will have an API of their own built on top of the basic API.</p>
|
||||
<p>The important advantage of having a range of APIs is not only about adapting it to the developer's capabilities but also about increasing the level of control you have over the code that partners embed into their apps:</p>
|
||||
<ol>
|
||||
<li>The apps that use physical interfaces are out of your control; for example, you can't force switching to newer versions of the platform or, let's say, add commercial inlets to them.</li>
|
||||
@ -3647,11 +3647,11 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>Code generation makes it possible to manipulate the desired form of integrations. For example, if our KPI is a number of searches performed through the API, we might alter the generated code so it will show the search panel in the most convenient position in the app; as partners using code-generation services rarely make any changes in the resulting code, and this will help us in reaching the goal.</li>
|
||||
<li>Finally, ready-to-use components and widgets are under your full control, and you might experiment with functionality exposed through them in partners' applications just as if it was your own service. (However, it doesn't automatically mean that you might draw some profits from having this control; for example, if you're allowing inserting pictures by their direct URL, your control over this integration is rather negligible, so it's generally better to provide those kinds of integration that allow having more control over the functionality in partners' apps.)</li>
|
||||
</ol>
|
||||
<p><strong>NB</strong>. While developing a “vertical” range of APIs, following the principles stated in the <a href="#back-compat-iceberg-waterline">“On the Waterline of the Iceberg”</a> chapter is crucial. You might manipulate widget content and behavior if, and only if, developers can't “escape the sandbox,” e.g. have direct access to low-level objects encapsulated within the widget.</p>
|
||||
<p><strong>NB</strong>. While developing a “vertical” range of APIs, following the principles stated in the <a href="#back-compat-iceberg-waterline">“On the Waterline of the Iceberg”</a> chapter is crucial. You might manipulate widget content and behavior if, and only if, developers can't “escape the sandbox,” i.e., have direct access to low-level objects encapsulated within the widget.</p>
|
||||
<p>In general, you should aim to have each partner using the API services in a manner that maximizes your profit as an API vendor. Where the partner doesn't try to make some unique experience and needs just a typical solution, you would benefit from making them use widgets, which are under your full control and thus ease the API version fragmentation problem and allow for experimenting in order to reach your KPIs. Where the partner possesses some unique expertise in the subject area and develops a unique service on top of your API, you would benefit from allowing full freedom in customizing the integration, so they might cover specific market niches and enjoy the advantage of offering more flexibility compared to services using competing APIs.</p><div class="page-break"></div><h3><a href="#api-product-kpi" class="anchor" id="api-product-kpi">Chapter 26. The API Key Performance Indicators</a><a href="#chapter-26" class="secondary-anchor" id="chapter-26"> </a></h3>
|
||||
<p>As we described in the previous chapters, there are many API monetization models, both direct and indirect. Importantly, most of them are fully or conditionally free for partners, and the direct-to-indirect benefits ratio tends to change during the API lifecycle. That naturally leads us to the question of how exactly shall we measure the API success and what goals are to be set for the product team.</p>
|
||||
<p>Of course, the most explicit metric is money: if your API is monetized directly or attracts visitors to a monetized service, the rest of the chapter will be of little interest to you, maybe just as a case study. If, however, the contribution of the API to the company's income cannot be simply measured, you have to stick to other, synthetic, indicators.</p>
|
||||
<p>The obvious key performance indicator (KPI) #1 is the number of end users and the number of integrations (i.e. partners using the API). Normally, they are in some sense a business health barometer: if there is a normal competitive situation among the API suppliers, and all of them are more or less in the same position, then the figure of how many developers (and consequently, how many end users) are using the API is the main metric of success of the API product.</p>
|
||||
<p>The obvious key performance indicator (KPI) #1 is the number of end users and the number of integrations (i.e., partners using the API). Normally, they are in some sense a business health barometer: if there is a normal competitive situation among the API suppliers, and all of them are more or less in the same position, then the figure of how many developers (and consequently, how many end users) are using the API is the main metric of success of the API product.</p>
|
||||
<p>However, sheer numbers might be deceiving, especially if we talk about free-to-use integrations. There are several factors that make them less reliable:</p>
|
||||
<ul>
|
||||
<li>
|
||||
@ -3672,9 +3672,9 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>the greater the API auditory is, the less the number of unique visitors means as at some moment the penetration will be close to 100%; for example, a regular Internet user interacts with Google or Facebook counters, well, every minute, so the daily audience of those API fundamentally cannot be increased further.</p>
|
||||
</li>
|
||||
</ul>
|
||||
<p>All the abovementioned problems naturally lead us to a very simple conclusion: not only the raw numbers of users and partners are to be gauged, but their engagement as well, e.g. the target actions (such as searching, observing some data, interacting with widgets) shall be determined and counted. Ideally, these target actions must correlate with the API monetization model:</p>
|
||||
<p>All the abovementioned problems naturally lead us to a very simple conclusion: not only the raw numbers of users and partners are to be gauged, but their engagement as well, i.e., the target actions (such as searching, observing some data, interacting with widgets) shall be determined and counted. Ideally, these target actions must correlate with the API monetization model:</p>
|
||||
<ul>
|
||||
<li>if the API is monetized through displaying ads, then the user's activity towards those ads (e.g. clicks, interactions) is to be measured;</li>
|
||||
<li>if the API is monetized through displaying ads, then the user's activity towards those ads (e.g., clicks, interactions) is to be measured;</li>
|
||||
<li>if the API attracts customers to the core service, then count the transitions;</li>
|
||||
<li>if the API is needed for collecting feedback and gathering UGC, then calculate the number of reviews left and entities edited.</li>
|
||||
</ul>
|
||||
@ -3688,8 +3688,8 @@ ProgramContext.dispatch = (action) => {
|
||||
</ul>
|
||||
<h4>SLA</h4>
|
||||
<p>This chapter would be incomplete if we didn't mention the “hygienic” KPI — the service level and the service availability. We won't be describing the concept in detail, as the API SLA isn't any different from any other digital services SLAs. Let us just state that this metric must be tracked, especially if we talk about pay-to-use APIs. However, in many cases, API vendors prefer to offer rather loose SLAs, treating the provided functionality as a data access or content licensing service.</p>
|
||||
<p>Still, let us re-iterate once more: any problems with your API are automatically multiplied by the number of partners you have, especially if the API is vital for them, e.g. the API outage makes the main functionality of their services unavailable. (And actually, because of the above-mentioned reasons, the average quality of integrations implies that partners' services will suffer even if the availability of the API is not formally speaking critical for them, but because developers use it excessively and do not bother with proper error handling.)</p>
|
||||
<p>It is important to mention that predicting the workload for the API service is rather complicated. Sub-optimal API usage, e.g. initializing the API in those application and website parts where it's not actually needed, might lead to a colossal increase in the number of requests after changing a single line of partner's code. The safety margin for an API service must be much higher than for a regular service for end users — it must survive the situation of the largest partner suddenly starting querying the API on every page and every application screen. (If the partner is already doing that, then the API must survive doubling the load if the partner by accident starts initializing the API twice on each page / screen.)</p>
|
||||
<p>Still, let us re-iterate once more: any problems with your API are automatically multiplied by the number of partners you have, especially if the API is vital for them, i.e., the API outage makes the main functionality of their services unavailable. (And actually, because of the above-mentioned reasons, the average quality of integrations implies that partners' services will suffer even if the availability of the API is not formally speaking critical for them, but because developers use it excessively and do not bother with proper error handling.)</p>
|
||||
<p>It is important to mention that predicting the workload for the API service is rather complicated. Sub-optimal API usage, e.g., initializing the API in those application and website parts where it's not actually needed, might lead to a colossal increase in the number of requests after changing a single line of partner's code. The safety margin for an API service must be much higher than for a regular service for end users — it must survive the situation of the largest partner suddenly starting querying the API on every page and every application screen. (If the partner is already doing that, then the API must survive doubling the load if the partner by accident starts initializing the API twice on each page / screen.)</p>
|
||||
<p>Another extremely important hygienic minimum is the informational security of the API service. In the worst-case scenario, namely, if an API service vulnerability allows for exploiting partner applications, one security loophole will in fact be exposed <em>in every partner application</em>. Needless to say that the cost of such a mistake might be overwhelmingly colossal, even if the API itself is rather trivial and has no access to sensitive data (especially if we talk about webpages where no “sandbox” for third-party scripts exists, and any piece of code might let's say track the data entered in forms). API services must provide the maximum protection level (for example, choose cryptographical protocols with a certain overhead) and promptly react to any reports regarding possible vulnerabilities.</p>
|
||||
<h4>Comparing to Competitors</h4>
|
||||
<p>While measuring KPIs of any service, it's important not only to evaluate your own numbers but also to match them against the state of the market:</p>
|
||||
@ -3701,7 +3701,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Getting answers to those questions might be quite non-trivial in the case of API services. Indeed, how could you learn how many integrations has your competitor had during the same period of time, and what number of target actions had happened on their platform? Sometimes, the providers of popular analytical tools might help you with this, but usually, you have to monitor the potential partners' apps and websites and gather the statistics regarding APIs they're using. The same applies to market research: unless your niche is significant enough for some analytical company to conduct a study, you will have to either commission such work or make your own estimations — conversely, through interviewing potential customers.</p><div class="page-break"></div><h3><a href="#api-product-antifraud" class="anchor" id="api-product-antifraud">Chapter 27. Identifying Users and Preventing Fraud</a><a href="#chapter-27" class="secondary-anchor" id="chapter-27"> </a></h3>
|
||||
<p>In the context of working with an API, we talk about two kinds of users of the system:</p>
|
||||
<ul>
|
||||
<li>users-developers, e.g. your partners writing code atop of the API;</li>
|
||||
<li>users-developers, i.e., your partners writing code atop of the API;</li>
|
||||
<li>end users interacting with applications implemented by the users-developers.</li>
|
||||
</ul>
|
||||
<p>In most cases, you need to have both of them identified (in a technical sense: discern one unique customer from another) to have answers to the following questions:</p>
|
||||
@ -3717,8 +3717,8 @@ ProgramContext.dispatch = (action) => {
|
||||
</ul>
|
||||
<p>In the case of commercial APIs, the quality and timeliness of gathering this data are twice that important, as the tariff plans (and therefore the entire business model) depend on it. Therefore, the question of <em>how exactly</em> we're identifying users is crucial.</p>
|
||||
<h4>Identifying Applications and Their Owners</h4>
|
||||
<p>Let's start with the first user category, e.g. API business partners-developers. The important remark: there are two different entities we must learn to identify, namely applications and their owners.</p>
|
||||
<p>An application is roughly speaking a logically separate case of API usage, usually — literally an application (mobile or desktop one) or a website, e.g. some technical entity. Meanwhile, an owner is a legal body that you have the API usage agreement signed. If API Terms of Service (ToS) imply different limits and/or tariffs depending on the type of the service or the way it uses the API, this automatically means the necessity to track one owner's applications separately.</p>
|
||||
<p>Let's start with the first user category, i.e., API business partners-developers. The important remark: there are two different entities we must learn to identify, namely applications and their owners.</p>
|
||||
<p>An application is roughly speaking a logically separate case of API usage, usually — literally an application (mobile or desktop one) or a website, i.e., some technical entity. Meanwhile, an owner is a legal body that you have the API usage agreement signed. If API Terms of Service (ToS) imply different limits and/or tariffs depending on the type of the service or the way it uses the API, this automatically means the necessity to track one owner's applications separately.</p>
|
||||
<p>In the modern world, the factual standard for identifying both entities is using API keys: a developer who wants to start using an API must obtain an API key bound to their contact info. Thus the key identifies the application while the contact data identifies the owner.</p>
|
||||
<p>Though this practice is universally widespread we can't but notice that in most cases it's useless, and sometimes just destructive.</p>
|
||||
<p>Its general advantage is the necessity to supply actual contact info to get a key, which theoretically allows for contacting the application owner if needed. (In the real world, it doesn't work: key owners often don't read mailboxes they provided upon registration; and if the owner is a company, it easily might be a no-one's mailbox or a personal email of some employee that left the company a couple of years ago.)</p>
|
||||
@ -3743,11 +3743,11 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Usually, you can put forward some requirements for self-identifying of partners, but asking end users to reveal contact information is impossible in most cases. All the methods of measuring the audience described below are imprecise and often heuristic. (Even if partner application functionality is only available after registration and you do have access to that profile data, it's still a game of assumptions, as an individual account is not the same as an individual user: several different persons might use a single account, or, vice versa, one person might register many accounts.) Also, note that gathering this sort of data might be legally regulated (though we will be mostly speaking about anonymized data, there might still be some applicable law).</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>The most simple and obvious indicator is an IP address. It's very hard to counterfeit them (e.g. the API server always knows the remote address), and the IP address statistics are reasonably demonstrative.</p>
|
||||
<p>The most simple and obvious indicator is an IP address. It's very hard to counterfeit them (i.e., the API server always knows the remote address), and the IP address statistics are reasonably demonstrative.</p>
|
||||
<p>If the API is provided as a server-to-server one, there will be no access to the end user's IP address. However, it makes sense to require partners to propagate the IP address (for example, in a form of the <code>X-Forwarded-For</code> header) — among other things, to help partners fight fraud and unintended usage of the API.</p>
|
||||
<p>Until recently, IP addresses were also a convenient statistics indicator because it was quite expensive to get a large pool of unique addresses. However, with ipv6 advancement this restriction is no longer actual; ipv6 rather put the light on the fact that you can't just count unique addresses — the aggregates are to be tracked:</p>
|
||||
<ul>
|
||||
<li>the cumulative number of requests by networks, e.g. the hierarchical calculations (the number of /8, /16, /24, etc. networks)</li>
|
||||
<li>the cumulative number of requests by networks, i.e., the hierarchical calculations (the number of /8, /16, /24, etc. networks)</li>
|
||||
<li>the cumulative statistics by autonomous networks (AS);</li>
|
||||
<li>the API requests through known public proxies and TOR network.</li>
|
||||
</ul>
|
||||
@ -3772,8 +3772,8 @@ ProgramContext.dispatch = (action) => {
|
||||
<p><em>Behavioral</em> analysis means we're examining the history of requests made by a specific user, searching for non-typical patterns, such as “unhuman” order of traversing endpoints or too small pauses between requests.</p>
|
||||
<p><strong>Importantly</strong>, when we talk about “users,” we will have to make duplicate systems to observe them both using tokens (cookies, logins, phone numbers) and IP addresses, as malefactors aren't obliged to preserve the tokens between requests, or might keep a pool of them to impede their exposure.</p>
|
||||
<h5><a href="#chapter-28-paragraph-2" id="chapter-28-paragraph-2" class="anchor">2. Requesting an Additional Authentication Factor</a></h5>
|
||||
<p>As both static and behavioral analyses are heuristic, it's highly desirable to not make decisions based solely on their outcome but rather ask the suspicious users to additionally prove they're making legitimate requests. If such a mechanism is in place, the quality of an anti-fraud system will be dramatically improved, as it allows for increasing system sensitivity and enabling pro-active defense, e.g. asking users to pass the tests in advance.</p>
|
||||
<p>In the case of services for end users, the main method of acquiring the second factor is redirecting to a captcha page. In the case of APIs it might be problematic, especially if you initially neglected the “Stipulate Restrictions” rule we've given in the <a href="#api-design-describing-interfaces">“Describing Final Interfaces”</a> chapter. In many cases, you will have to impose this responsibility on partners (e.g. it will be partners who show captchas and identify users based on the signals received from the API endpoints). This will, of course, significantly impair the convenience of working with the API.</p>
|
||||
<p>As both static and behavioral analyses are heuristic, it's highly desirable to not make decisions based solely on their outcome but rather ask the suspicious users to additionally prove they're making legitimate requests. If such a mechanism is in place, the quality of an anti-fraud system will be dramatically improved, as it allows for increasing system sensitivity and enabling pro-active defense, e.g., asking users to pass the tests in advance.</p>
|
||||
<p>In the case of services for end users, the main method of acquiring the second factor is redirecting to a captcha page. In the case of APIs it might be problematic, especially if you initially neglected the “Stipulate Restrictions” rule we've given in the <a href="#api-design-describing-interfaces">“Describing Final Interfaces”</a> chapter. In many cases, you will have to impose this responsibility on partners (i.e., it will be partners who show captchas and identify users based on the signals received from the API endpoints). This will, of course, significantly impair the convenience of working with the API.</p>
|
||||
<p><strong>NB</strong>. Instead of captcha, there might be other actions introducing additional authentication factors. It might be the phone number confirmation or the second step of the 3D-Secure protocol. The important part is that requesting an additional authentication step must be stipulated in the program interface, as it can't be added later in a backwards-compatible manner.</p>
|
||||
<p>Other popular mechanics of identifying robots include offering a bait (“honeypot”) or employing the execution environment checks (starting from rather trivial ones like executing JavaScript on the webpage and ending with sophisticated techniques of checking application integrity checksums).</p>
|
||||
<h5><a href="#chapter-28-paragraph-3" id="chapter-28-paragraph-3" class="anchor">3. Restricting Access</a></h5>
|
||||
@ -3789,8 +3789,8 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>The third option is the most effective one in technical terms as it allows to put the ball in the malefactor's court: it is now them who need to invent how to learn if the robot was detected. But from the moral point of view (and from the legal perspective as well) this method is rather questionable, especially if we take into account the probability of false-positive signals, meaning that some real users will get the fake data.</p>
|
||||
<p>Thereby, you have only one method that really works: filing complaints to hosting providers, ISPs, or law enforcement authorities. Needless to say, this brings certain reputational risks, and the reaction time is rather not lightning fast.</p>
|
||||
<p>In most cases, you're not fighting fraud — you're actually increasing the cost of the attack, simultaneously buying yourself enough time to make administrative moves against the perpetrator. Preventing API misusage completely is impossible as malefactors might ultimately employ the expensive but bulletproof solution — to hire real people to make the requests to the API on real devices through legitimate applications.</p>
|
||||
<p>An opinion exists, which the author of this book shares, that engaging in this sword-against-shield confrontation must be carefully thought out, and advanced technical solutions are to be enabled only if you are one hundred percent sure it is worth it (e.g. if they steal real money or data). By introducing elaborate algorithms, you rather conduct an evolutional selection of the smartest and most cunning cybercriminals, counteracting to whom will be way harder than to those who just naïvely call API endpoints with <code>curl</code>. What is even more important, in the final phase — e.g. when filing the complaint to authorities — you will have to prove the alleged ToS violation, and doing so against an advanced fraudster will be problematic. So it's rather better to have all the malefactors monitored (and regularly complained against), and escalate the situation (e.g. enable the technical protection and start legal actions) only if the threat passes a certain threshold. That also implies that you must have all the tools ready, and just keep them below fraudsters' radars.</p>
|
||||
<p>Out of the author of this book's experience, the mind games with malefactors, when you respond to any improvement of their script with the smallest possible effort that is enough to break it, might continue indefinitely. This strategy, e.g. making fraudsters guess which traits were used to ban them this time (instead of unleashing the whole heavy artillery potential), annoys amateur “hackers” greatly as they lack hard engineering skills and just give up eventually.</p>
|
||||
<p>An opinion exists, which the author of this book shares, that engaging in this sword-against-shield confrontation must be carefully thought out, and advanced technical solutions are to be enabled only if you are one hundred percent sure it is worth it (e.g., if they steal real money or data). By introducing elaborate algorithms, you rather conduct an evolutional selection of the smartest and most cunning cybercriminals, counteracting to whom will be way harder than to those who just naïvely call API endpoints with <code>curl</code>. What is even more important, in the final phase — i.e., when filing the complaint to authorities — you will have to prove the alleged ToS violation, and doing so against an advanced fraudster will be problematic. So it's rather better to have all the malefactors monitored (and regularly complained against), and escalate the situation (i.e., enable the technical protection and start legal actions) only if the threat passes a certain threshold. That also implies that you must have all the tools ready, and just keep them below fraudsters' radars.</p>
|
||||
<p>Out of the author of this book's experience, the mind games with malefactors, when you respond to any improvement of their script with the smallest possible effort that is enough to break it, might continue indefinitely. This strategy, i.e., making fraudsters guess which traits were used to ban them this time (instead of unleashing the whole heavy artillery potential), annoys amateur “hackers” greatly as they lack hard engineering skills and just give up eventually.</p>
|
||||
<h4>Dealing with Stolen Keys</h4>
|
||||
<p>Let's now move to the second type of unlawful API usage, namely using in the malefactor's applications keys stolen from conscientious partners. As the requests are generated by real users, captcha won't help, though other techniques will.</p>
|
||||
<ol>
|
||||
@ -3887,14 +3887,14 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>First, you need to determine whether this service covers your needs in general (as quickly as possible);</li>
|
||||
<li>If it does, you look for specific functionality to resolve your specific case.</li>
|
||||
</ol>
|
||||
<p>In fact, newcomers (e.g. those developers who are not familiar with the API) usually want just one thing: to assemble the code that solves their problem out of existing code samples and never return to this issue again. Sounds not exactly reassuringly, given the amount of work invested into the API and its documentation development, but that's what the reality looks like. Also, that's the root cause of developers' dissatisfaction with the docs: it's literally impossible to have articles covering exactly that problem the developer comes with being detailed exactly to the extent the developer knows the API concepts. In addition, non-newcomers (e.g. those developers who have already learned the basics concepts and are now trying to solve some advanced problems) do not need these “mixed examples” articles as they look for some deeper understanding.</p>
|
||||
<p>In fact, newcomers (i.e., those developers who are not familiar with the API) usually want just one thing: to assemble the code that solves their problem out of existing code samples and never return to this issue again. Sounds not exactly reassuringly, given the amount of work invested into the API and its documentation development, but that's what the reality looks like. Also, that's the root cause of developers' dissatisfaction with the docs: it's literally impossible to have articles covering exactly that problem the developer comes with being detailed exactly to the extent the developer knows the API concepts. In addition, non-newcomers (i.e., those developers who have already learned the basics concepts and are now trying to solve some advanced problems) do not need these “mixed examples” articles as they look for some deeper understanding.</p>
|
||||
<h4>Introductory Notes</h4>
|
||||
<p>Documentation frequently suffers from being excessively clerical; it's being written using formal terminology (which often requires reading the glossary before the actual docs) and is frequently unreasonably inflated. So instead of a two-word answer to a user's question, a couple of paragraphs is conceived — a practice we strongly disapprove of. The perfect documentation must be simple and laconic, and all the terms must be either explained in the text or given a reference to such an explanation. However, “simple” doesn't mean “illiterate”: remember, the documentation is the face of your product, so grammar errors and improper usage of terms are unacceptable.</p>
|
||||
<p>Also, keep in mind that documentation will be used for searching as well, so every page should contain all the keywords required to be properly ranked by search engines. Unfortunately, this requirement contradicts the simple-and-laconic principle; that's the way.</p>
|
||||
<h4>Documentation Content Types</h4>
|
||||
<h5><a href="#chapter-30-paragraph-1" id="chapter-30-paragraph-1" class="anchor">1. Specification / Reference</a></h5>
|
||||
<p>Any documentation starts with a formal functional description. This content type is the most inconvenient to use, but you must provide it. A reference is the hygienic minimum of the API documentation. If you don't have a doc that describes all methods, parameters, options, variable types, and their allowed values, then it's not an API but amateur dramatics.</p>
|
||||
<p>Today, a reference must be also a machine-readable specification, e.g. comply with some standard, for example, OpenAPI.</p>
|
||||
<p>Today, a reference must be also a machine-readable specification, i.e., comply with some standard, for example, OpenAPI.</p>
|
||||
<p>The specification must comprise not only formal descriptions but implicit agreements as well, such as the event generation order or unobvious side-effects of the API methods. Its important applied value is advisory consulting: developers will refer to it to clarify unobvious situations.</p>
|
||||
<p><strong>Importantly</strong>, formal specification <em>is not documentation</em> per se. The documentation is <em>the words you write</em> in the descriptions of each field and method. Without them, the specification might be used just for checking whether your namings are fine enough for developers to guess their meaning.</p>
|
||||
<p>Today, the method nomenclature descriptions are frequently additionally exposed as ready-to-use request collections or code fragments for Postman or analogous tools.</p>
|
||||
@ -3905,19 +3905,19 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>examples must be laconic and atomic: mixing a bunch of tricks in one code sample dramatically reduces its readability and applicability;</li>
|
||||
<li>examples must be close to real-world app code; the author of this book once faced a situation when a synthetic code sample, totally meaningless in the real world, was mindlessly replicated by developers in abundance.</li>
|
||||
</ul>
|
||||
<p>Ideally, examples should be linked to all other kinds of documentation, i.e. the reference should contain code samples relevant to the entity being described.</p>
|
||||
<p>Ideally, examples should be linked to all other kinds of documentation, e.g., the reference might contain code samples relevant to the entity being described.</p>
|
||||
<h5><a href="#chapter-30-paragraph-3" id="chapter-30-paragraph-3" class="anchor">3. Sandboxes</a></h5>
|
||||
<p>Code samples will be much more useful to developers if they are “live,” e.g. provided as editable pieces of code that might be modified and executed. In the case of library APIs, the online sandbox featuring a selection of code samples will suffice, and existing online services like JSFiddle might be used. With other types of APIs, developing sandboxes might be much more complicated:</p>
|
||||
<p>Code samples will be much more useful to developers if they are “live,” i.e., provided as editable pieces of code that might be modified and executed. In the case of library APIs, the online sandbox featuring a selection of code samples will suffice, and existing online services like JSFiddle might be used. With other types of APIs, developing sandboxes might be much more complicated:</p>
|
||||
<ul>
|
||||
<li>if the API provides access to some data, then the sandbox must allow working with a real dataset, either a developer's own one (e.g. bound to their user profile) or some test data;</li>
|
||||
<li>if the API provides access to some data, then the sandbox must allow working with a real dataset, either a developer's own one (e.g., bound to their user profile) or some test data;</li>
|
||||
<li>if the API provides an interface, visual or programmatic, to some non-online environment, like UI libs for mobile devices do, then the sandbox itself must be an emulator or a simulator of that environment, in a form of an online service or a standalone app.</li>
|
||||
</ul>
|
||||
<h5><a href="#chapter-30-paragraph-4" id="chapter-30-paragraph-4" class="anchor">4. Tutorial</a></h5>
|
||||
<p>A tutorial is a specifically written human-readable text describing some concepts of working with the API. A tutorial is something in-between a reference and examples. It implies some learning, more thorough than copy-pasting code samples, but requires less time investment than reading the whole reference.</p>
|
||||
<p>A tutorial is a sort of “book” that you write to explain to the reader how to work with your API. So, a proper tutorial must follow book-writing patterns, e.g. explain the concepts coherently and consecutively chapter after chapter. Also, a tutorial must provide:</p>
|
||||
<p>A tutorial is a sort of “book” that you write to explain to the reader how to work with your API. So, a proper tutorial must follow book-writing patterns, i.e., explain the concepts coherently and consecutively chapter after chapter. Also, a tutorial must provide:</p>
|
||||
<ul>
|
||||
<li>general knowledge of the subject area; for example, a tutorial for cartographical APIs must explain trivia regarding geographical coordinates and working with them;</li>
|
||||
<li>proper API usage scenarios, e.g. the “happy paths”;</li>
|
||||
<li>proper API usage scenarios, i.e., the “happy paths”;</li>
|
||||
<li>proper reactions to program errors that could happen;</li>
|
||||
<li>detailed studies on advanced API functionality (with detailed examples).</li>
|
||||
</ul>
|
||||
@ -3934,7 +3934,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>both questions and answers must be formulated clearly and succinctly; it's acceptable (and even desirable) to provide links to corresponding reference and tutorial articles, but the answer itself can't be longer than a couple of paragraphs.</li>
|
||||
</ul>
|
||||
<p>Also, FAQs are a convenient place to explicitly highlight the advantages of the API. In a question-answer form, you might demonstrably show how your API solves complex problems easily and handsomely. (Or at least, <em>solves them</em>, unlike the competitors' products.)</p>
|
||||
<p>If technical support conversations are public, it makes sense to store all the questions and answers as a separate service to form a knowledge base, e.g. a set of “real-life” questions and answers.</p>
|
||||
<p>If technical support conversations are public, it makes sense to store all the questions and answers as a separate service to form a knowledge base, i.e., a set of “real-life” questions and answers.</p>
|
||||
<h5><a href="#chapter-30-paragraph-6" id="chapter-30-paragraph-6" class="anchor">6. Offline Documentation</a></h5>
|
||||
<p>Though we live in the online world, an offline version of the documentation (in a form of a generated doc file) still might be useful — first of all, as a snapshot of the API specification valid for a specific date.</p>
|
||||
<h4>Content Duplication Problems</h4>
|
||||
@ -3948,11 +3948,11 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>The problem becomes worse if you're supporting not only different API versions but also different environments / platforms / programming languages; for example, if your UI lib supports both iOS and Android. Then both documentation versions are equal, and it's impossible to pessimize one of them.</p>
|
||||
<p>In this case, you need to choose one of the following strategies:</p>
|
||||
<ul>
|
||||
<li>if the documentation topic content is totally identical for every platform, e.g. only the code syntax differs, you will need to develop generalized documentation: each article provides code samples (and maybe some additional notes) for every supported platform on a single page;</li>
|
||||
<li>if the documentation topic content is totally identical for every platform, i.e., only the code syntax differs, you will need to develop generalized documentation: each article provides code samples (and maybe some additional notes) for every supported platform on a single page;</li>
|
||||
<li>on the contrary, if the content differs significantly, as is in the iOS/Android case, we might suggest splitting the documentation sites (up to having separate domains for each platform): the good news is that developers almost always need one specific version, and they don't care about other platforms.</li>
|
||||
</ul>
|
||||
<h4>The Documentation Quality</h4>
|
||||
<p>The best documentation happens when you start viewing it as a product in the API product range, e.g. begin analyzing customer experience (with specialized tools), collect and process feedback, set KPIs and work on improving them.</p>
|
||||
<p>The best documentation happens when you start viewing it as a product in the API product range, i.e., begin analyzing customer experience (with specialized tools), collect and process feedback, set KPIs and work on improving them.</p>
|
||||
<h4>Was This Article Helpful to You?</h4>
|
||||
<p><a href="https://forms.gle/WPdQ9KsJt3fxqpyw6">Yes / No</a></p><div class="page-break"></div><h3><a href="#api-product-testing" class="anchor" id="api-product-testing">Chapter 31. The Testing Environment</a><a href="#chapter-31" class="secondary-anchor" id="chapter-31"> </a></h3>
|
||||
<p>If the operations executed via the API imply consequences for end users or partners (cost money, in particular) you must provide a test version of the API. In this testing API, real-world actions either don't happen at all (for instance, orders are created but nobody serves them) or are simulated by cheaper means (let's say, instead of sending an SMS to a user, an email is sent to the developer's mailbox).</p>
|
||||
@ -3970,7 +3970,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>The disadvantage of this approach is that client developers still need to know how the “flip side” of the system works, though in simplified terms.</p>
|
||||
<h5><a href="#chapter-31-paragraph-2" id="chapter-31-paragraph-2" class="anchor">2. The Simulator of Pre-Defined Scenarios</a></h5>
|
||||
<p>The alternative to providing the testing environment API is simulating the working scenarios. In this case, the testing environment takes control over “underwater” parts of the system and “plays out” all external agents' actions. In our coffee example, that means that, after the order is submitted, the system will simulate all the preparation steps and then the delivery of the beverage to the customer.</p>
|
||||
<p>The advantage of this approach is that it demonstrates vividly how the system works according to the API vendor design plans, e.g. in which sequence the events are generated, and which stages the order passes through. It also reduces the chance of making mistakes in testing scripts, as the API vendor guarantees the actions will be executed in the correct order with the right parameters.</p>
|
||||
<p>The advantage of this approach is that it demonstrates vividly how the system works according to the API vendor design plans, e.g., in which sequence the events are generated, and which stages the order passes through. It also reduces the chance of making mistakes in testing scripts, as the API vendor guarantees the actions will be executed in the correct order with the right parameters.</p>
|
||||
<p>The main disadvantage is the necessity to create a separate scenario for each unhappy path (effectively, for every possible error), and give developers the capability of denoting which scenario they want to run. (For example, like that: if there is a pre-agreed comment to the order, the system will simulate a specific error, and developers will be able to write and debug the code that deals with the error.)</p>
|
||||
<h4>The Automation of Testing</h4>
|
||||
<p>Your final goal in implementing testing APIs, regardless of which option you choose, is allowing partners to automate the QA process for their products. The testing environment should be developed with this purpose in mind; for example, if an end user might be brought to a 3-D Secure page to pay for the order, the testing environment API must provide some way of simulating the successful (or not) passing of this step. Also, in both variants, it's possible (and desirable) to allow running the scenarios in a fast-forward manner that will allow making auto-testing much faster than manual testing.</p>
|
||||
|
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -1 +1,23 @@
|
||||
### On Design Patterns in the API Context
|
||||
### [On Design Patterns in the API Context][api-patterns-context]
|
||||
|
||||
The concept of [“Patterns”](https://en.wikipedia.org/wiki/Software_design_pattern#History) in the field of software engineering was introduced by Kent Beck and Ward Cunningham in 1987 and popularized by “The Gang of Four” (Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides) in their book “Design Patterns: Elements of Reusable Object-Oriented Software,” which was published in 1994. According to the most widespread definition, a software design pattern is a “general, reusable solution to a commonly occurring problem within a given context.”
|
||||
|
||||
If we talk about APIs, especially those to which developers are end users (e.g., frameworks or operating system interfaces), the classical software design patterns are well applicable to them. Indeed, many examples in the previous Section of this book are just about applying some design patterns.
|
||||
|
||||
However, if we try to extend this approach to include API development in general, we will soon find that many typical API design issues are high-level and can't be reduced to basic software patterns. Let's say, caching resources (and invalidating the cache) or organizing paginated access are not covered in classical writings.
|
||||
|
||||
In this Section, we will specify those API design problems that we see as the most important ones. We are not aiming to encompass *every* problem, let alone every solution, and rather focus on describing approaches to solving typical problems with their pros and cons. We do understand that readers familiar with the works of “The Gang of Four,” Grady Booch, and Martin Fowler might expect a more systematic approach and greater depth of outreach from a section called “The API Patterns,” and we apologize to them in advance.
|
||||
|
||||
**NB**: the first such pattern we need to mention is the API-first approach to software engineering, which we [described in the corresponding chapter](#intro-api-first-approach).
|
||||
|
||||
#### The Fundamentals of Solving Typical API Design Problems
|
||||
|
||||
Before we proceed to the patterns, we need to formulate first, how developing APIs differs from developing other kinds of software. Below, we will formulate three important concepts, which we will be referring to in the subsequent chapters.
|
||||
|
||||
1. The more distributed and multi-faceted systems are and the more general-purpose channels of communication used, the more errors occur in the process of interaction. In the most interesting case of distributed many-layered client-server systems, raising an exception on client side (like losing context as a result of app crash and restart), server side (the pipeline of executing a query threw at some stage), communication channel (connection fully or partially lost), or any other interim agent (intermediate web-server hasn't got a response from backend and returned a gateway error) is a norm of life, and all systems must be designed in a manner that **in a case of exception of any kind, API clients must be able to restore their state** and continue operating normally.
|
||||
|
||||
2. The more partners use the API, the more chance is that some of the mechanisms of the expected workflow are implemented wrongly. In other words, **not only physical mistakes related to network or server overload should be expected, but also logical ones caused by improper API usage** (and, in particular, there should be safeguards to avoid errors in one partner's code leading to a denial of service for other partners).
|
||||
|
||||
3. Any part of the system might introduce unpredictable latencies to serving requests, and those latencies might be quite high — seconds and tens of seconds. Even if you fully control the execution environment and network, client apps are fully capable of impeding themselves as they might be written in a suboptimal manner or be executed on a low-performant or overloaded device. Because of that, **proper API design must not rely on critical operations being executed quickly**. This includes:
|
||||
* if carrying out some task through API requires making a sequence of calls, there must be a mechanism for resuming the operation from the current step if needed instead of re-starting it from the beginning;
|
||||
* operations affecting shared resources should impose some locking mechanisms for the operation duration.
|
@ -1 +1,23 @@
|
||||
### О паттернах проектирования в контексте API
|
||||
### О паттернах проектирования в контексте API
|
||||
|
||||
Концепция [«паттернов»](https://en.wikipedia.org/wiki/Software_design_pattern#History) в области разработки программного обеспечения была введёна Кентом Бэком и Уордом Каннингемом в 1987 году, и популяризирован «бандой четырёх» (Эрих Гамма, Ричард Хелм, Ральф Джонсон и Джон Влиссидес) в их книге «Приёмы объектно-ориентированного проектирования. Паттерны проектирования», изданной в 1994 году. Согласно общепринятому определению, паттерны программирования — «повторяемая архитектурная конструкция, представляющая собой решение проблемы проектирования в рамках некоторого часто возникающего контекста».
|
||||
|
||||
Если мы говорим об API, особенно если конечным потребителем этих API является разработчик (интерфейсы фреймворков, операционных систем), классические паттерны проектирования вполне к ним применимы. И действительно, многие из описанных в предыдущем разделе примеров представляют собой применение того или иного паттерна.
|
||||
|
||||
Однако, если мы попытаемся обобщить этот подход на разработку API в целом, то увидим, что большинство типичных проблем дизайна API являются более высокоуровневыми и не сводятся к базовым паттернам разработки ПО. Скажем, проблемы кэширования ресурсов (и инвалидации кэша) или организация пагинации классиками не покрыты.
|
||||
|
||||
В рамках этого раздела мы попытаемся описать те задачи проектирования API, которые представляются нам наиболее важными. Мы не претендуем здесь на то, чтобы охватить *все* проблемы и тем более — все решения, и скорее фокусируемся на описании подходов к решению типовых задач с их достоинствами и недостатками. Мы понимаем, что читатель, знакомый с классическими трудами «банды четырёх», Гради Буча и Мартина Фаулера ожидает от раздела с названием «Паттерны API» большей системности и ширины охвата, и заранее просим у него прощения.
|
||||
|
||||
**NB**: первый паттерн, о котором необходимо упомянуть — это API-first подход к разработке ПО, который мы описали [в соответствующей главе](#intro-api-first-approach).
|
||||
|
||||
#### Принципы решения типовых проблем проектирования API
|
||||
|
||||
Прежде, чем излагать сами паттерны, нам нужно понять, чем же разработка API отличается от разработки обычных приложений. Ниже мы сформулируем три важных принципа, на которые будем ссылаться в последующих главах.
|
||||
|
||||
1. Чем более распределена и многосоставна система, чем более общий канал связи используется для коммуникации — тем более вероятны ошибки в процессе взаимодействия. В частности, в наиболее интересном нам кейсе распределённых многослойных клиент-серверных систем возникновение исключения на клиенте (например, потеря контекста в результате перезапуска приложения), на сервере (конвейер выполнения запроса выбросил исключение на каком-то шаге), в канале связи (соединение полностью или частично потеряно) или любом промежуточном агенте (например, промежуточный веб-сервер не дождался ответа бэкенда и вернул ошибку гейтвея) — норма жизни, и все системы должны проектироваться таким образом, что **в случае возникновения исключения любого рода клиенты API должны быть способны восстановить своё состояние** и продолжить корректно работать.
|
||||
|
||||
2. Чем больше различных партнёров подключено к API, тем больше вероятность того, что какие-то из предусмотренных вами механизмов обеспечения корректности взаимодействия будет имплементирован неправильно. Иными словами, **вы должны ожидать не только физических ошибок, связанных с состоянием сети или перегруженностью сервера, но и логических, связанных с неправильным использованием API** (и, в частности, предотвращать возможный отказ в обслуживании одних партнёров из-за ошибок в коде других партнёров).
|
||||
|
||||
3. Любая из частей системы может вносить непредсказуемые задержки исполнения запросов, причём достаточно высокого — секунды, десятки секунд — порядка. Даже если вы полностью контролируете среду исполнения и сеть, задержку может вносить само клиентское приложение, которое может быть просто написано неоптимальным образом или же работать на слабом или перегруженном устройстве. Поэтому **при проектировании API нельзя полагаться на то, что критические действия выполнятся быстро**. В частности:
|
||||
* для операций, состоящих из нескольких шагов, необходимо предусматривать возможность при необходимости продолжить выполнение с текущего шага, а не с начала;
|
||||
* для операций с разделяемыми ресурсами необходимо предусматривать механизмы блокировки ресурса.
|
@ -1,23 +0,0 @@
|
||||
### О паттернах проектирования в контексте API
|
||||
|
||||
Термин [«паттерны»](https://en.wikipedia.org/wiki/Software_design_pattern#History) применительно к разработке программного обеспечения был введён Кентом Бэком и Уордом Каннингемом в 1987 году, и популяризирован «бандой четырёх» (Эрих Гамма, Ричард Хелм, Ральф Джонсон и Джон Влиссидес) в их книге «Приёмы объектно-ориентированного проектирования. Паттерны проектирования», изданной в 1994 году. Согласно общепринятому определению, паттерны программирования — «повторяемая архитектурная конструкция, представляющая собой решение проблемы проектирования в рамках некоторого часто возникающего контекста».
|
||||
|
||||
Если мы говорим об API, особенно если конечным потребителем этих API является разработчик (интерфейсы фреймворков, операционных систем), классические паттерны проектирования вполне к ним применимы. И действительно, многие из описанных в предыдущем разделе примеров представляют собой применение того или иного паттерна.
|
||||
|
||||
Однако, если мы попытаемся обобщить этот опыт на разработку API в целом, то увидим, что большинство типичных проблем дизайна API являются более высокоуровневыми и не сводятся к базовым паттернам разработки ПО. Скажем, проблемы кэширования ресурсов (и инвалидации кэша) или организация пагинации классиками не покрыты.
|
||||
|
||||
В рамках этого раздела мы попытаемся описать те задачи проектирования API и подходы к их решению, которые представляются нам наиболее важными. Мы не претендуем здесь на то, чтобы охватить *все* проблемы и тем более — все решения, и скорее фокусируемся на описании подходов к решению типовых задач с их достоинствами и недостатками. Мы понимаем, что читатель, знакомый с классическими трудами «банды четырёх», Гради Буча и Мартина Фаулера ожидает от раздела с названием «Паттерны API» большей системности и ширины охвата, и заранее просим у него прощения.
|
||||
|
||||
**NB**: первые два паттерна, о которых необходимо упомянуть — это API-first подход к разработке ПО и организация аутентификации пользователей, которые мы описали во введении.
|
||||
|
||||
#### Принципы решения типовых проблем проектирования API
|
||||
|
||||
Прежде, чем излагать сами паттерны, нам нужно понять, чем же разработка API отличается от разработки обычных приложений. Ниже мы сформулируем три важных принципа, на которые будем ссылаться в последующих главах.
|
||||
|
||||
Рассматривая API как мост, связывающий два разных программных контекста, мы в большинстве случаев ожидаем, что стороны каньона функционируют независимо и изолированно друг от друга — причём чем крупнее контексты и сложнее каналы передачи данных, тем более независимо и изолированно они работают. В частности:
|
||||
|
||||
1. Чем более распределена и многосоставна система, чем более общий канал связи используется для коммуникации — тем более вероятны ошибки в процессе взаимодействия. В частности, в наиболее интересном нам кейсе распределённых многослойных клиент-серверных систем возникновение исключения на клиенте (потеря контекста, т.е. перезапуск приложения), на сервере (конвейер выполнения запроса выбросил исключение на каком-то шаге), в канале связи (соединение полностью или частично потеряно) или любом промежуточном агенте (например, промежуточный веб-сервер не дождался ответа бэкенда и вернул ошибку гейтвея) — норма жизни, и все системы должны проектироваться таким образом, что **в случае возникновения исключения любого рода клиенты API должны быть способны восстановить своё состояние** и продолжить корректно работать.
|
||||
|
||||
2. Чем больше различных партнёров подключено к API, тем больше вероятность того, что какие-то из предусмотренных вами механизмов обеспечения корректности взаимодействия будет имплементирован неправильно. Иными словами, **вы должны ожидать не только физических ошибок, связанных с состоянием сети или перегруженностью сервера, но и логических, связанных с неправильным использованием API** (и, в частности, предотвращать возможный отказ в обслуживании одних партнёров из-за ошибок в коде других партнёров).
|
||||
|
||||
3. Любая из частей системы может вносить непредсказуемые задержки исполнения запросов, причём достаточно высокого — секунды, десятки секунд — порядка. Даже если вы полностью контролируете среду исполнения и сеть, задержку может вносить само клиентское приложение, которое может быть просто написано неоптимальным образом или же работать на слабом или перегруженном устройстве. Если выполнение какой-то задачи через API требует последовательного исполнения цепочки вызовов, то **необходимо предусматривать механизмы синхронизации промежуточного состояния**.
|
Loading…
x
Reference in New Issue
Block a user