mirror of
https://github.com/twirl/The-API-Book.git
synced 2025-01-17 17:44:13 +02:00
style fix & clarifications
This commit is contained in:
parent
0299b0ab7e
commit
f48e8f857c
BIN
docs/API.en.epub
BIN
docs/API.en.epub
Binary file not shown.
131
docs/API.en.html
131
docs/API.en.html
@ -2494,7 +2494,7 @@ X-Idempotency-Token: <idempotency token>
|
||||
<p><strong>NB</strong>: in this book, we often use short identifiers like "123" in code examples; that's for the convenience of reading the book on small screens. Do not replicate this practice in a real-world API.</p>
|
||||
<h5><a href="#chapter-11-paragraph-23" id="chapter-11-paragraph-23" class="anchor">23. Stipulate Future Restrictions</a></h5>
|
||||
<p>With the API popularity growth, it will inevitably become necessary to introduce technical means of preventing illicit API usage, such as displaying captchas, setting honeypots, raising the “too many requests” exceptions, installing anti-DDoS proxies, etc. All these things cannot be done if the corresponding errors and messages were not described in the docs from the very beginning.</p>
|
||||
<p>You are not obliged to actually generate those exceptions, but you might stipulate this possibility in the terms of service. For example, you might describe the <code>429 Too Many Requests</code> error or captcha redirect but implement the functionality when it's actually needed.</p>
|
||||
<p>You are not obliged to actually generate those exceptions, but you might stipulate this possibility in the Terms of Service (ToS). For example, you might describe the <code>429 Too Many Requests</code> error or captcha redirect but implement the functionality when it's actually needed.</p>
|
||||
<p>It is extremely important to leave room for multi-factored authentication (such as TOTP, SMS, or 3D-secure-like technologies) if it's possible to make payments through the API. In this case, it's a must-have from the very beginning.</p>
|
||||
<h5><a href="#chapter-11-paragraph-24" id="chapter-11-paragraph-24" class="anchor">24. No Bulk Access to Sensitive Data</a></h5>
|
||||
<p>If it's possible to get through the API users' personal data, bank card numbers, private messages, or any other kind of information, exposing which might seriously harm users, partners, and/or you — there must be <em>no</em> methods of bulk getting the data, or at least there must be rate limiters, page size restrictions, and, ideally, multi-factored authentication in front of them.</p>
|
||||
@ -3624,7 +3624,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Different companies employ different approaches to determining the granularity of API services, e.g. what is counted as a separate product and what is not. To some extent, this is a matter of convenience and taste judgment. Consider splitting an API into parts if:</p>
|
||||
<ul>
|
||||
<li>it makes sense for partners to integrate only one API part, e.g. some isolated subset of the API provides enough means to solve users' problems;</li>
|
||||
<li>API parts might be versioned separately and independently, and it is meaningful from the partners' point of view (this usually means that those “isolated” APIs are either fully independent or maintain strict backwards compatibility and introduce new major versions only when it's absolutely necessary; otherwise, maintaining a matrix which API No. 1 version is compatible with which API No. 2 version will soon become a catastrophe);</li>
|
||||
<li>API parts might be versioned separately and independently, and it is meaningful from the partners' point of view (this usually means that those “isolated” APIs are either fully independent or maintain strict backwards compatibility and introduce new major versions only when it's absolutely necessary; otherwise, maintaining a matrix which API #1 version is compatible with which API #2 version will soon become a catastrophe);</li>
|
||||
<li>it makes sense to set tariffs and limits for each API service independently;</li>
|
||||
<li>the auditory of the API segments (either developers, business owners, or end users) is not overlapping, and “selling” granular API to customers is much easier than aggregated.</li>
|
||||
</ul>
|
||||
@ -3651,16 +3651,19 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>In general, you should aim to have each partner using the API services in a manner that maximizes your profit as an API vendor. Where the partner doesn't try to make some unique experience and needs just a typical solution, you would benefit from making them use widgets, which are under your full control and thus ease the API version fragmentation problem and allow for experimenting in order to reach your KPIs. Where the partner possesses some unique expertise in the subject area and develops a unique service on top of your API, you would benefit from allowing full freedom in customizing the integration, so they might cover specific market niches and enjoy the advantage of offering more flexibility compared to services using competing APIs.</p><div class="page-break"></div><h3><a href="#api-product-kpi" class="anchor" id="api-product-kpi">Chapter 26. The API Key Performance Indicators</a><a href="#chapter-26" class="secondary-anchor" id="chapter-26"> </a></h3>
|
||||
<p>As we described in the previous chapters, there are many API monetization models, both direct and indirect. Importantly, most of them are fully or conditionally free for partners, and the direct-to-indirect benefits ratio tends to change during the API lifecycle. That naturally leads us to the question of how exactly shall we measure the API success and what goals are to be set for the product team.</p>
|
||||
<p>Of course, the most explicit metric is money: if your API is monetized directly or attracts visitors to a monetized service, the rest of the chapter will be of little interest to you, maybe just as a case study. If, however, the contribution of the API to the company's income cannot be simply measured, you have to stick to other, synthetic, indicators.</p>
|
||||
<p>The obvious key performance indicator (KPI) No. 1 is the number of end users and the number of integrations (i.e. partners using the API). Normally, they are in some sense a business health barometer: if there is a normal competitive situation among the API suppliers, and all of them are more or less in the same position, then the figure of how many developers (and consequently, how many end users) are using the API is the main metric of the success of the API product.</p>
|
||||
<p>However, sheer numbers might be deceiving, especially if we talk about free-to-use integrations. There are several factors that twist the statistics:</p>
|
||||
<p>The obvious key performance indicator (KPI) #1 is the number of end users and the number of integrations (i.e. partners using the API). Normally, they are in some sense a business health barometer: if there is a normal competitive situation among the API suppliers, and all of them are more or less in the same position, then the figure of how many developers (and consequently, how many end users) are using the API is the main metric of success of the API product.</p>
|
||||
<p>However, sheer numbers might be deceiving, especially if we talk about free-to-use integrations. There are several factors that make them less reliable:</p>
|
||||
<ul>
|
||||
<li>
|
||||
<p>the high-level API services that are meant for straightforward integration (see the previous chapter) are significantly distorting the statistics if the competitors don't provide such services; typically, for one full-scale integration there will be tens, maybe hundreds, of those lightweight embedded widgets; thereby, it's crucial to have partners counted for each kind of the integration independently;</p>
|
||||
<p>the high-level API services that are meant for point-and-click integration (see the previous chapter) are significantly distorting the statistics, especially if the competitors don't provide such services; typically, for one full-scale integration there will be tens, maybe hundreds, of those lightweight embedded widgets;</p>
|
||||
<ul>
|
||||
<li>thereby, it's crucial to have partners counted for each kind of the integration independently;</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<p>partners tend to use the API in suboptimal ways:</p>
|
||||
<ul>
|
||||
<li>embed it at every site page / application screen instead of only those where end users can really interact with the API;</li>
|
||||
<li>embed it at every website page / application screen instead of only those where end users can really interact with the API;</li>
|
||||
<li>put widgets somewhere deep in the page / screen footer, or hide it behind spoilers;</li>
|
||||
<li>initialize a broad range of API modules, but use only a limited subset of them;</li>
|
||||
</ul>
|
||||
@ -3684,10 +3687,10 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>the definition of target action depends on the monetization model and might be quite straightforward (like the number of paying partners, or the number of paid ad clicks) or, to the contrary, pretty implicit (like the growth of the company's developer blog auditory).</li>
|
||||
</ul>
|
||||
<h4>SLA</h4>
|
||||
<p>This chapter would be incomplete if we didn't mention the “hygienic” KPI — the service level and the service availability. We won't be describing the concept in detail, as the API SLA isn't any different from any other digital services SLAs. Let us point out that this metric must be tracked, especially if we talk about pay-to-use APIs. However, in many cases, API vendors prefer to offer rather loose SLAs, treating the provided functionality as data access or a content licensing service.</p>
|
||||
<p>This chapter would be incomplete if we didn't mention the “hygienic” KPI — the service level and the service availability. We won't be describing the concept in detail, as the API SLA isn't any different from any other digital services SLAs. Let us just state that this metric must be tracked, especially if we talk about pay-to-use APIs. However, in many cases, API vendors prefer to offer rather loose SLAs, treating the provided functionality as a data access or content licensing service.</p>
|
||||
<p>Still, let us re-iterate once more: any problems with your API are automatically multiplied by the number of partners you have, especially if the API is vital for them, e.g. the API outage makes the main functionality of their services unavailable. (And actually, because of the above-mentioned reasons, the average quality of integrations implies that partners' services will suffer even if the availability of the API is not formally speaking critical for them, but because developers use it excessively and do not bother with proper error handling.)</p>
|
||||
<p>It is important to mention that predicting the workload for the API service is rather complicated. Sub-optimal API usage, e.g. initializing the API in those application and website parts where it's not actually needed, might lead to a colossal increase in the number of requests after changing a single line of partner's code. The safety margin for an API service must be much higher than for a regular service for end users — it must survive the situation of the largest partner suddenly starting querying the API on every page and every application screen. (If the partner is already doing that, then the API must survive doubling the load: imagine the partner accidentally starts initializing the API twice on each page / screen.)</p>
|
||||
<p>Another extremely important hygienic minimum is the informational security of the API service. In the worst-case scenario, namely, if an API service vulnerability allows for exploiting partner applications, one security loophole will in fact be exposed <em>in every partner application</em>. Needless to say that the cost of such a mistake might be overwhelmingly colossal, even if the API itself is rather trivial and has no access to sensitive data (especially if we talk about webpages where no “sandbox” for third-party scripts exists, and any piece of code might let's say track the data entered in forms). API services must provide the maximum protection level (for example, choose cryptographical protocols with a certain overhead) and promptly react to any messages regarding possible vulnerabilities.</p>
|
||||
<p>It is important to mention that predicting the workload for the API service is rather complicated. Sub-optimal API usage, e.g. initializing the API in those application and website parts where it's not actually needed, might lead to a colossal increase in the number of requests after changing a single line of partner's code. The safety margin for an API service must be much higher than for a regular service for end users — it must survive the situation of the largest partner suddenly starting querying the API on every page and every application screen. (If the partner is already doing that, then the API must survive doubling the load if the partner by accident starts initializing the API twice on each page / screen.)</p>
|
||||
<p>Another extremely important hygienic minimum is the informational security of the API service. In the worst-case scenario, namely, if an API service vulnerability allows for exploiting partner applications, one security loophole will in fact be exposed <em>in every partner application</em>. Needless to say that the cost of such a mistake might be overwhelmingly colossal, even if the API itself is rather trivial and has no access to sensitive data (especially if we talk about webpages where no “sandbox” for third-party scripts exists, and any piece of code might let's say track the data entered in forms). API services must provide the maximum protection level (for example, choose cryptographical protocols with a certain overhead) and promptly react to any reports regarding possible vulnerabilities.</p>
|
||||
<h4>Comparing to Competitors</h4>
|
||||
<p>While measuring KPIs of any service, it's important not only to evaluate your own numbers but also to match them against the state of the market:</p>
|
||||
<ul>
|
||||
@ -3695,13 +3698,13 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>is your service growing faster than the market itself or is the rate the same, or is it even less?</li>
|
||||
<li>what proportion of the growth is caused by the growth of the market, and what is related to your efforts?</li>
|
||||
</ul>
|
||||
<p>Getting answers to those questions might be quite non-trivial in the case of API services. Indeed, how could you learn how many integrations has your competitor had during the same period of time, and what number of target actions had happened on their platform? Sometimes, the providers of popular analytical tools might help you with this, but usually, you have to monitor the potential partners' apps and websites and gather the statistics regarding APIs they're using. The same applies to market research: unless your niche is significant enough for some analytical company to conduct market research, you will have to either commission such a study or make your own estimations — conversely, through interviewing potential customers.</p><div class="page-break"></div><h3><a href="#api-product-antifraud" class="anchor" id="api-product-antifraud">Chapter 27. Identifying Users and Preventing Fraud</a><a href="#chapter-27" class="secondary-anchor" id="chapter-27"> </a></h3>
|
||||
<p>Getting answers to those questions might be quite non-trivial in the case of API services. Indeed, how could you learn how many integrations has your competitor had during the same period of time, and what number of target actions had happened on their platform? Sometimes, the providers of popular analytical tools might help you with this, but usually, you have to monitor the potential partners' apps and websites and gather the statistics regarding APIs they're using. The same applies to market research: unless your niche is significant enough for some analytical company to conduct a study, you will have to either commission such work or make your own estimations — conversely, through interviewing potential customers.</p><div class="page-break"></div><h3><a href="#api-product-antifraud" class="anchor" id="api-product-antifraud">Chapter 27. Identifying Users and Preventing Fraud</a><a href="#chapter-27" class="secondary-anchor" id="chapter-27"> </a></h3>
|
||||
<p>In the context of working with an API, we talk about two kinds of users of the system:</p>
|
||||
<ul>
|
||||
<li>users-developers, e.g. your partners writing code atop of the API;</li>
|
||||
<li>end users interacting with applications implemented by the users-developers.</li>
|
||||
</ul>
|
||||
<p>In most cases, you need to have both of them identified (in a technical sense: discern one unique visitor from another) to have answers to the following questions:</p>
|
||||
<p>In most cases, you need to have both of them identified (in a technical sense: discern one unique customer from another) to have answers to the following questions:</p>
|
||||
<ul>
|
||||
<li>how many users are interacting with the system (simultaneously, daily, monthly, and yearly)?</li>
|
||||
<li>how many actions does each user make?</li>
|
||||
@ -3715,22 +3718,22 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>In the case of commercial APIs, the quality and timeliness of gathering this data are twice that important, as the tariff plans (and therefore the entire business model) depend on it. Therefore, the question of <em>how exactly</em> we're identifying users is crucial.</p>
|
||||
<h4>Identifying Applications and Their Owners</h4>
|
||||
<p>Let's start with the first user category, e.g. API business partners-developers. The important remark: there are two different entities we must learn to identify, namely applications and their owners.</p>
|
||||
<p>An application is roughly speaking a logically separate case of API usage, usually — literally an application (mobile or desktop one) or a website, e.g. some technical entity. Meanwhile, an owner is a legal body that you have the API usage agreement signed with, e.g. a legal body. if API tariffs imply some limits and/or tariffs depend on the type of the service or the way it uses the API, this automatically means the necessity to track one owner's applications separately.</p>
|
||||
<p>An application is roughly speaking a logically separate case of API usage, usually — literally an application (mobile or desktop one) or a website, e.g. some technical entity. Meanwhile, an owner is a legal body that you have the API usage agreement signed. If API Terms of Service (ToS) imply different limits and/or tariffs depending on the type of the service or the way it uses the API, this automatically means the necessity to track one owner's applications separately.</p>
|
||||
<p>In the modern world, the factual standard for identifying both entities is using API keys: a developer who wants to start using an API must obtain an API key bound to their contact info. Thus the key identifies the application while the contact data identifies the owner.</p>
|
||||
<p>Though this practice is universally widespread we can't but notice that in most cases it's useless, and sometimes just destructive.</p>
|
||||
<p>Its general advantage is the necessity to supply actual contact info to get a key, which theoretically allows for contacting the application owner if needed. (In the real world, it doesn't work: key owners often don't read mailboxes they provided upon registration; and if the owner is a company, it easily might be a no-one's mailbox or a personal email of some employee that left the company a couple of years ago.)</p>
|
||||
<p>The main disadvantage of using API keys is that they <em>don't</em> allow for reliably identifying both applications and their owners.</p>
|
||||
<p>If there are free limits to API usage, there is a temptation to obtain many API keys bound to different owners to fit those free limits. You may raise the bar of having such multi-accounts by requiring, let's say, providing a phone number or bank card data, but there are popular services for automatically issuing both. Paying for a virtual SIM or credit card (to say nothing about buying the stolen ones) will always be cheaper than paying the proper API tariff — unless it's the API for creating those cards. Therefore, API key-based user identification (if you're not requiring the physical contract to be signed) does not mean you don't need to double-check whether users comply with the terms of service and do not issue several keys for one app.</p>
|
||||
<p>Another problem is that an API key might be simply stolen from a lawful partner; in the case of client or web applications, that's quite trivial.</p>
|
||||
<p>It might look like the problem is not that important in the case of server-to-server integrations, but it actually is. Imagine that a partner provides a public service of their own that uses your API under the hood. That usually means there is an endpoint in the partner's backend that performs a request to the API and returns the result, and this endpoint perfectly suits as a free replacement of the API access to a cybercriminal. Of course, you might say this fraud is a problem of partners', but, first, it would be naïve to expect each partner develops their own anti-fraud system, and, second, it's just sub-optimal: obviously, a centralized anti-fraud system would be way more effective than a bunch of amateur implementations. Also, server keys might also be stolen: it's much harder than stealing client keys but doable. With any popular API, sooner or later you will face the situation of stolen keys made available to the public (or a key owner just shares it with acquaintances out of the kindness of their heart).</p>
|
||||
<p>Another problem is that an API key might be simply stolen from a lawful partner; in the case of mobile or web applications, that's quite trivial.</p>
|
||||
<p>It might look like the problem is not that important in the case of server-to-server integrations, but it actually is. Imagine that a partner provides a public service of their own that uses your API under the hood. That usually means there is an endpoint in the partner's backend that performs a request to the API and returns the result, and this endpoint perfectly suits as a free replacement of direct access to the API for a cybercriminal. Of course, you might say this fraud is a problem of partners', but, first, it would be naïve to expect each partner develops their own anti-fraud system, and, second, it's just sub-optimal: obviously, a centralized anti-fraud system would be way more effective than a bunch of amateur implementations. Also, server keys might also be stolen: it's much harder than stealing client keys but doable. With any popular API, sooner or later you will face the situation of stolen keys made available to the public (or a key owner just shared it with acquaintances out of the kindness of their heart).</p>
|
||||
<p>One way or another, a problem of independent validation arises: how can we control whether the API endpoint is requested by a user in compliance with the terms of service?</p>
|
||||
<p>Mobile applications might be conveniently tracked through their identifiers in the corresponding store (Google Play, App Store, etc.), so it makes sense to require this identifier to be passed by partners as an API initialization parameter. Websites with some degree of confidence might be identified by the Referer and Origin HTTP headers.</p>
|
||||
<p>Mobile applications might be conveniently tracked through their identifiers in the corresponding store (Google Play, App Store, etc.), so it makes sense to require this identifier to be passed by partners as an API initialization parameter. Websites with some degree of confidence might be identified by the <code>Referer</code> and <code>Origin</code> HTTP headers.</p>
|
||||
<p>This data is not itself reliable, but it allows for making cross-checks:</p>
|
||||
<ul>
|
||||
<li>if the key was issued for one specific domain but requests are coming with a different Referer, it makes sense to investigate the situation and maybe ban the possibility to access the API with this Referer or this key;</li>
|
||||
<li>if an application initializes API by providing the key registered to another application, it makes sense to contact the store administration and ask for removing one of the apps.</li>
|
||||
<li>if a key was issued for one specific domain but requests are coming with a different <code>Referer</code>, it makes sense to investigate the situation and maybe ban the possibility to access the API with this <code>Referer</code> or this key;</li>
|
||||
<li>if an application initializes API by providing a key registered to another application, it makes sense to contact the store administration and ask for removing one of the apps.</li>
|
||||
</ul>
|
||||
<p><strong>NB</strong>: don't forget to set infinite limits for using the API with the <code>localhost</code>, <code>127.0.0.1</code> / <code>[::1]</code> Referers, and also for your own sandbox if it exists. Yes, abusers will sooner or later learn this fact and will start using it, but otherwise, you will ban local development and your own website much sooner than that.</p>
|
||||
<p><strong>NB</strong>: don't forget to set infinite limits for using the API with the <code>localhost</code>, <code>127.0.0.1</code> / <code>[::1]</code> <code>Referer</code>s, and also for your own sandbox if it exists. Yes, abusers will sooner or later learn this fact and will start exploiting it, but otherwise, you will ban local development and your own website much sooner than that.</p>
|
||||
<p>The general conclusion is:</p>
|
||||
<ul>
|
||||
<li>it is highly desirable to have partners formally identified (either through obtaining API keys or by providing contact data such as website domain or application identifier in a store while initializing the API);</li>
|
||||
@ -3741,21 +3744,23 @@ ProgramContext.dispatch = (action) => {
|
||||
<ol>
|
||||
<li>
|
||||
<p>The most simple and obvious indicator is an IP address. It's very hard to counterfeit them (e.g. the API server always knows the remote address), and the IP address statistics are reasonably demonstrative.</p>
|
||||
<p>If the API is provided as a server-to-server one, there will be no access to the end user's IP address. However, it makes sense to require partners to propagate the IP address (for example, in a form of the <code>X-Forwarder-For</code> header) — among other things, to help partners fight fraud and unintended usage of the API.</p>
|
||||
<p>Until recently, IP addresses were also a convenient statistics indicator because it was quite expensive to get a large pool of unique addresses. However, with ipv6 advancement this restriction is no longer actual; ipv6 rather put the light on the fact that you can't just count unique addresses — the aggregates are to be tracked:
|
||||
* the cumulative number of requests by networks, e.g. the hierarchical calculations (the number of /8, /16, /24, etc. networks)
|
||||
* the cumulative statistics by autonomous networks (AS);
|
||||
* the API requests through known public proxies and TOR network.</p>
|
||||
<p>An abnormal number of requests in one network might be evidence of the API being actively used inside some corporative environment (or in this region NATs are widespread).</p>
|
||||
<p>If the API is provided as a server-to-server one, there will be no access to the end user's IP address. However, it makes sense to require partners to propagate the IP address (for example, in a form of the <code>X-Forwarded-For</code> header) — among other things, to help partners fight fraud and unintended usage of the API.</p>
|
||||
<p>Until recently, IP addresses were also a convenient statistics indicator because it was quite expensive to get a large pool of unique addresses. However, with ipv6 advancement this restriction is no longer actual; ipv6 rather put the light on the fact that you can't just count unique addresses — the aggregates are to be tracked:</p>
|
||||
<ul>
|
||||
<li>the cumulative number of requests by networks, e.g. the hierarchical calculations (the number of /8, /16, /24, etc. networks)</li>
|
||||
<li>the cumulative statistics by autonomous networks (AS);</li>
|
||||
<li>the API requests through known public proxies and TOR network.</li>
|
||||
</ul>
|
||||
<p>An abnormal number of requests in one network might be evidence of the API being actively used inside some corporative environment (or NATs being widespread in the region).</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Additional means of tracking are users' unique identifiers, most notably cookies. However, most recently this method of gathering data is under attack from several sides: browser makers restrict third-party cookies, users are employing anti-tracker software, and lawmakers started to roll out legal requirements against data collection. In the current situation, it's much easier to drop cookie usage than to be compliant with all the regulations.</p>
|
||||
<p>Additional means of tracking are users' unique identifiers, most notably cookies. However, most recently this method of gathering data got attacked from several directions: browser makers restrict third-party cookies, users are employing anti-tracker software, and lawmakers started to roll out legal requirements against data collection. In the current situation, it's much easier to drop cookie usage than to be compliant with all the regulations.</p>
|
||||
<p>All this leads to a situation when public APIs (especially those installed on free-to-use sites and applications) are very limited in the means of collecting statistics and analyzing user behavior. And that impacts not only fighting all kinds of fraud but analyzing use cases as well. That's the way.</p>
|
||||
</li>
|
||||
</ol>
|
||||
<p><strong>NB</strong>. In some jurisdictions, IP addresses are considered personal data, and collecting them is prohibited as well. We don't dare to advise on how an API vendor might at the same time be able to fight prohibited content on the platform and don't have access to users' IP addresses. We presume that complying with such legislation implies storing statistics by IP address hashes. (And just in case we won't mention that building a rainbow table for SHA-256 hashes covering the entire 4-billion range of IPv4 addresses would take several hours on a regular office-grade computer.)</p><div class="page-break"></div><h3><a href="#api-product-tos-violations" class="anchor" id="api-product-tos-violations">Chapter 28. The Technical Means of Preventing ToS Violations</a><a href="#chapter-28" class="secondary-anchor" id="chapter-28"> </a></h3>
|
||||
<p>Implementing the paradigm of a centralized system of preventing partner endpoints-bound fraud, which we described in the previous chapter, in practice faces non-trivial difficulties.</p>
|
||||
<p>The task of filtering out illicit API requests generally comprises three steps:</p>
|
||||
<p>The task of filtering out illicit API requests comprises three steps:</p>
|
||||
<ul>
|
||||
<li>identifying suspicious users;</li>
|
||||
<li>optionally, asking for an additional authentication factor;</li>
|
||||
@ -3763,40 +3768,40 @@ ProgramContext.dispatch = (action) => {
|
||||
</ul>
|
||||
<h5><a href="#chapter-28-paragraph-1" id="chapter-28-paragraph-1" class="anchor">1. Identifying Suspicious Users</a></h5>
|
||||
<p>Generally speaking, there are two approaches we might take, the static one and the dynamic (behavioral) one.</p>
|
||||
<p><em>Statically</em> we monitor suspicions activity surges, as described in the previous chapter, marking an unusually high density of requests coming from specific networks or Referer's (actually, <em>any</em> piece of information suits if it divides users into more or less independent groups: for example, OS version or system language if you can gather those).</p>
|
||||
<p><em>Statically</em> we monitor suspicions activity surges, as described in the previous chapter, marking an unusually high density of requests coming from specific networks or <code>Referer</code>s (actually, <em>any</em> piece of information suits if it splits users into more or less independent groups: for example, OS version or system language would suffice if you can gather those).</p>
|
||||
<p><em>Behavioral</em> analysis means we're examining the history of requests made by a specific user, searching for non-typical patterns, such as “unhuman” order of traversing endpoints or too small pauses between requests.</p>
|
||||
<p><strong>Importantly</strong>, when we talk about “user,” we will have to make a second analytical contour to work with IP addresses, as malefactors aren't obliged to preserve cookies or other identification tokens, or will keep a pool of such tokens to impede their exposure.</p>
|
||||
<p><strong>Importantly</strong>, when we talk about “users,” we will have to make duplicate systems to observe them both using tokens (cookies, logins, phone numbers) and IP addresses, as malefactors aren't obliged to preserve the tokens between requests, or might keep a pool of them to impede their exposure.</p>
|
||||
<h5><a href="#chapter-28-paragraph-2" id="chapter-28-paragraph-2" class="anchor">2. Requesting an Additional Authentication Factor</a></h5>
|
||||
<p>As both static and behavioral analyses are heuristic, it's highly desirable to not make decisions based solely on their outcome but rather ask the suspicious users to additionally prove they're making legitimate requests. If such a mechanism is in place, the quality of an anti-fraud system will be dramatically improved, as it allows for increasing system sensitivity and enabling pro-active defense, e.g. asking users to pass the tests in advance.</p>
|
||||
<p>In the case of services for end users, the main method of acquiring the second factor is redirecting to a captcha page. In the case of APIs it might be problematic, especially if you initially neglected the “Stipulate Restrictions” rule we've given in the <a href="#api-design-describing-interfaces">“Describing Final Interfaces”</a> chapter. In many cases, you will have to impose this responsibility on partners (e.g. it will be partners who show captchas and identify users based on the signals received from the API endpoints). This will, of course, significantly impair the convenience of working with the API.</p>
|
||||
<p><strong>NB</strong>. Instead of captcha, there might be other actions introducing additional authentication factors. It might be the phone number confirmation or the second step of the 3D-Secure protocol. The important part is that requesting an additional authentication step must be stipulated in the program interface, as it can't be added later in a backwards-compatible manner.</p>
|
||||
<p>Other popular mechanics of identifying robots include offering a bait (“honeypot”) or employing the execution environment checks (starting from rather trivial like executing JavaScript on the webpage and ending with sophisticated techniques of checking application integrity).</p>
|
||||
<p>Other popular mechanics of identifying robots include offering a bait (“honeypot”) or employing the execution environment checks (starting from rather trivial ones like executing JavaScript on the webpage and ending with sophisticated techniques of checking application integrity checksums).</p>
|
||||
<h5><a href="#chapter-28-paragraph-3" id="chapter-28-paragraph-3" class="anchor">3. Restricting Access</a></h5>
|
||||
<p>The illusion of having a broad choice of technical means of identifying fraud users should not deceive you as you will soon discover the lack of effective methods of restricting those users. Banning them by cookie / Referer / User-Agent makes little to no impact as this data is supplied by clients, and might be easily forged. In the end, you have four mechanisms for suppressing illegal activities:</p>
|
||||
<p>The illusion of having a broad choice of technical means of identifying fraud users should not deceive you as you will soon discover the lack of effective methods of restricting those users. Banning them by cookie / <code>Referer</code> / <code>User-Agent</code> makes little to no impact as this data is supplied by clients, and might be easily forged. In the end, you have four mechanisms for suppressing illegal activities:</p>
|
||||
<ul>
|
||||
<li>banning users by IP (networks, autonomous systems);</li>
|
||||
<li>requiring mandatory user identification (maybe layered: login / login with confirmed phone number / login with confirmed identity / login with confirmed identity and biometrics / etc.);</li>
|
||||
<li>returning fake responses;</li>
|
||||
<li>banning users by IP (networks, autonomous systems)</li>
|
||||
<li>requiring mandatory user identification (maybe tiered: login / login with confirmed phone number / login with confirmed identity / login with confirmed identity and biometrics / etc.)</li>
|
||||
<li>returning fake responses</li>
|
||||
<li>filing administrative abuse reports.</li>
|
||||
</ul>
|
||||
<p>The problem with option number one is the collateral damage you will inflict, especially if you have to ban subnets.</p>
|
||||
<p>The problem with the first option is the collateral damage you will inflict, especially if you have to ban subnets.</p>
|
||||
<p>The second option, though quite rational, is usually inapplicable to real APIs, as not every partner will agree with the approach, and definitely not every end user. This will also require being compliant with the existing personal data laws.</p>
|
||||
<p>The third option is the most effective one in technical terms as it allows to put the ball in the malefactor's court: it is now them who need to invent how to learn if the robot was detected. But from the moral point of view (and from the legal perspective as well) this method is rather questionable, especially if we take into account the probability of false-positive signals, meaning that some real users will get the fake data.</p>
|
||||
<p>Thereby, you have only one method that really works: filing complaints to hosting providers, ISPs, or law enforcement authorities. Needless to say, this brings certain reputational risks, and the reaction time is rather not lightning fast.</p>
|
||||
<p>In most cases, you're not fighting fraud — you're actually increasing the cost of the attack, simultaneously buying yourself enough time to make administrative moves against the perpetrator. Preventing API misusage completely is impossible as malefactors might ultimately employ the expensive but bulletproof solution — to hire real people to make the requests to the API on real devices through legal applications.</p>
|
||||
<p>An opinion exists, which the author of this book shares, that engaging in this sword-against-shield confrontation must be carefully thought out, and advanced technical solutions are to be enabled only if you are one hundred percent sure it is worth it (e.g. if they steal real money or data). By introducing elaborate algorithms, you rather conduct an evolutional selection of the smartest and most cunning cybercriminals, counteracting who will be way harder than those who just naively call API endpoints with <code>curl</code>. What is even more important, in the final phase — e.g. when filing the complaint to authorities — you will have to prove the alleged ToS violation, and doing so against an advanced fraudster will be problematic. So it's rather better to have all the malefactors monitored (and regularly complained against), and escalate the situation (e.g. enable the technical protection and start legal actions) only if the threat passes a certain threshold. That also implies that you must have all the tools ready, and just keep them below fraudsters' radars.</p>
|
||||
<p>In most cases, you're not fighting fraud — you're actually increasing the cost of the attack, simultaneously buying yourself enough time to make administrative moves against the perpetrator. Preventing API misusage completely is impossible as malefactors might ultimately employ the expensive but bulletproof solution — to hire real people to make the requests to the API on real devices through legitimate applications.</p>
|
||||
<p>An opinion exists, which the author of this book shares, that engaging in this sword-against-shield confrontation must be carefully thought out, and advanced technical solutions are to be enabled only if you are one hundred percent sure it is worth it (e.g. if they steal real money or data). By introducing elaborate algorithms, you rather conduct an evolutional selection of the smartest and most cunning cybercriminals, counteracting to whom will be way harder than to those who just naïvely call API endpoints with <code>curl</code>. What is even more important, in the final phase — e.g. when filing the complaint to authorities — you will have to prove the alleged ToS violation, and doing so against an advanced fraudster will be problematic. So it's rather better to have all the malefactors monitored (and regularly complained against), and escalate the situation (e.g. enable the technical protection and start legal actions) only if the threat passes a certain threshold. That also implies that you must have all the tools ready, and just keep them below fraudsters' radars.</p>
|
||||
<p>Out of the author of this book's experience, the mind games with malefactors, when you respond to any improvement of their script with the smallest possible effort that is enough to break it, might continue indefinitely. This strategy, e.g. making fraudsters guess which traits were used to ban them this time (instead of unleashing the whole heavy artillery potential), annoys amateur “hackers” greatly as they lack hard engineering skills and just give up eventually.</p>
|
||||
<h4>Dealing with Stolen Keys</h4>
|
||||
<p>Let's now move to the second type of unlawful API usage, namely using in the malefactor's applications keys stolen from conscientious partners. As the requests are generated by real users, captcha won't help, though other techniques will.</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>Maintaining metrics collection by IP addresses and subnets might be of use in this case as well. If the malefactor's app isn't a public one but rather targeted to some closed audience, this fact will be visible in the dashboards (and if you're lucky enough, you might also find suspicious Referers, public access to which is restricted).</p>
|
||||
<p>Maintaining metrics collection by IP addresses and subnets might be of use in this case as well. If the malefactor's app isn't a public one but rather targeted to some closed audience, this fact will be visible in the dashboards (and if you're lucky enough, you might also find suspicious <code>Referer</code>s, public access to which is restricted).</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Allowing partners to restrict the functionality available under specific API keys:</p>
|
||||
<ul>
|
||||
<li>
|
||||
<p>setting the allowed IP address range for server-to-server APIs, allowed Referer's and application ids for client APIs;</p>
|
||||
<p>setting the allowed IP address range for server-to-server APIs, allowed <code>Referer</code>s and application ids for client APIs;</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>white-listing only allowed API functions for a specific key;</p>
|
||||
@ -3813,7 +3818,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>for example, if on the partner's website, there is a form displaying the best lungo offers, for which the partners call the API endpoint like <code>/v1/search?recipe=lungo&api_key={apiKey}</code>, then the API key might be replaced with a signature like <code>sign = HMAC("recipe=lungo", apiKey)</code>; the signature might be stolen as well, but it will be useless for malefactors as they will be able to find only lungo with it;</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>instead of API keys, time-based one-time passwords (TOTP) might be used; these tokens are valid for a short period of time (usually, one minute) only, which makes working with stealing keys much more sophisticated.</p>
|
||||
<p>instead of API keys, time-based one-time passwords (TOTP) might be used; these tokens are valid for a short period of time only (typically, one minute), which makes using stolen keys much more complicated.</p>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
@ -3824,7 +3829,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Banning compromised API keys; the partners' reaction will be, of course, negative, but ultimately every business will prefer temporary disabling of some functionality over getting a multi-million bill.</p>
|
||||
</li>
|
||||
</ol><div class="page-break"></div><h3><a href="#api-product-customer-support" class="anchor" id="api-product-customer-support">Chapter 29. Supporting customers</a><a href="#chapter-29" class="secondary-anchor" id="chapter-29"> </a></h3>
|
||||
<p>First of all, an important remark: when we talk about supporting API customers, we mean supporting developers and to some extent business partners. End users seldom interact with APIs directly, with an exception of several non-standard cases.</p>
|
||||
<p>From banning users, let's change the topic to supporting them. First of all, an important remark: when we talk about supporting API customers, we mean helping developers and to some extent business partners. End users seldom interact with APIs directly, with an exception of several non-standard cases:</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>If you can't reach partners that are using the API incorrectly, you might have to display errors that end users can see. This might happen if the API was provided for free and with minimum partner identification requirements while in the growth phase, and then the conditions changed (a popular API version is no longer supported or became paid).</p>
|
||||
@ -3844,28 +3849,28 @@ ProgramContext.dispatch = (action) => {
|
||||
</ul>
|
||||
<p>The former is of course extremely important for any healthy service (including APIs) but again bears little API-related specifics. In the context of this book, we are much more interested in the latter.</p>
|
||||
<p>As an API is a program product, developers will be in fact asking how this specific piece of code that they have written works. This fact raises the level of required customer support staff members' expertise quite high as you need a software engineer to read the code and understand the problem. But this is but half of the problem; another half is, as we have mentioned in the previous chapters, that most of these questions will be asked by inexperienced or amateur developers. In a case of a popular API, it means that 9 out of 10 inquiries <em>will not be about the API</em>. Less skilled developers lack language knowledge, their experience with the platform is fragmented, and they can't properly formulate their problem (and therefore search for an answer on the Internet before contacting support; though, let us be honest, they usually don't even try).</p>
|
||||
<p>There are several options for tackling these issues.</p>
|
||||
<p>There are several options for tackling these issues:</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>The most user-friendly scenario is hiring people with basic technical skills as the first line of support. These employees must possess enough expertise in understanding how the API works to be able to identify those unrelated questions and respond to them according to some FAQ, point out to the relevant external resource (let's say, the support service of the platform or the community forum of the programming language), or redirect relevant issues to the API developers.</p>
|
||||
<p>The most user-friendly scenario is hiring people with basic technical skills as the first line of support. These employees must possess enough expertise in understanding how the API works to be able to identify those unrelated questions and respond to them according to some FAQ, point out to a relevant external resource (let's say, the support service of the OS or the community forum of the programming language) if the problem is not related to the API itself, and redirect relevant issues to the API developers.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>The inverse scenario: partners must pay for technical support, and it's the API developers who answer the questions. It doesn't actually make a significant difference in terms of the quality of the issues (it's still inexperienced developers who use the API and ask questions; you will just cut off those who can't afford paid support) but at least you won't have a hiring problem as you might allow yourself the luxury of hiring engineers for the first line of support.</p>
|
||||
<p>The inverse scenario: partners must pay for technical support, and it's the API developers who answer the questions. It doesn't actually make a significant difference in terms of the quality of the issues (it's still mostly inexperienced developers who can't solve the problem on their own; you will just cut off those who can't afford paid support) but at least you won't have a hiring problem as you might allow yourself the luxury of having engineers for the first line of support.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Partly (or, sometimes, fully) the developer community might help with solving the amateur problems (see the <a href="#api-product-devrel">“Communicating with Developers”</a> chapter). Usually, community members are pretty capable of answering those questions, especially if moderators help them.</p>
|
||||
</li>
|
||||
</ol>
|
||||
<p>Importantly, whatever options you choose, it's still the API developers in the second line of support simply because only they can fully understand the problem and the partners' code. That implies two important consequences.</p>
|
||||
<p>Importantly, whatever options you choose, it's still the API developers in the second line of support simply because only they can fully understand the problem and the partners' code. That implies two important consequences:</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>You must consider the time needed to investigate inquiries while planning the API development team time. Reading unfamiliar code and remote debugging are very hard and exhausting tasks. The more functionality you expose and the more platforms you support, the more load is put on the team in terms of dealing with support tickets.</p>
|
||||
<p>You must take into account working with inquiries while planning the API development team time. Reading unfamiliar code and remote debugging are very hard and exhausting tasks. The more functionality you expose and the more platforms you support, the more load is put on the team in terms of dealing with support tickets.</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>As a rule, developers are totally not happy with the perspective of sorting the incoming requests and answering them. The first line of support will still let through a lot of dilettante or badly formulated questions, and that will annoy on-duty API developers. There are several approaches to mitigate the problem:</p>
|
||||
<p>As a rule, developers are totally not happy about the perspective of coping with incoming requests and answering them. The first line of support will still let through a lot of dilettante or badly formulated questions, and that will annoy on-duty API developers. There are several approaches to mitigate the problem:</p>
|
||||
<ul>
|
||||
<li>
|
||||
<p>try to find people with a customer-oriented mindset, who like this activity, and encourage them (including financial stimulus) to deal with support; it might be someone on the team (and not necessarily a developer) or some active community member;</p>
|
||||
<p>try to find people with a customer-oriented mindset, who like this activity, and encourage them (including financial stimulus) to perform support functions; it might be someone on the team (and not necessarily a developer) or some active community member;</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>the remaining load must be distributed among the developers equally and fairly, up to introducing the duty calendar.</p>
|
||||
@ -3875,7 +3880,7 @@ ProgramContext.dispatch = (action) => {
|
||||
</ol>
|
||||
<p>And of course, analyzing the questions is a useful exercise to populate FAQs and improve the documentation and the first-line support scripts.</p>
|
||||
<h4>External Platforms</h4>
|
||||
<p>Sooner or later, you will find that customers ask their questions not only through the official channels, but also on numerous Internet-based forums, starting from those specifically created for this, like StackOverflow, and ending with social networks and personal blogs. It's up to you whether to spend time searching for such inquiries; we would rather recommend providing support through those sites that have convenient tools for that (like subscribing to specific tags).</p><div class="page-break"></div><h3><a href="#api-product-documentation" class="anchor" id="api-product-documentation">Chapter 30. The Documentation</a><a href="#chapter-30" class="secondary-anchor" id="chapter-30"> </a></h3>
|
||||
<p>Sooner or later, you will find that customers ask their questions not only through the official channels, but also on numerous Internet-based forums, starting from those specifically created for this, like StackOverflow, and ending with social networks and personal blogs. It's up to you whether to spend time searching for such inquiries. We would rather recommend providing support through those platforms that have convenient tools for that (like subscribing to specific tags).</p><div class="page-break"></div><h3><a href="#api-product-documentation" class="anchor" id="api-product-documentation">Chapter 30. The Documentation</a><a href="#chapter-30" class="secondary-anchor" id="chapter-30"> </a></h3>
|
||||
<p>Regretfully, many API providers pay miserable attention to the quality of documentation. Meanwhile, the documentation is the face of the product and the entry point to it. The problem becomes even worse if we acknowledge that it's almost impossible to write the help docs the developers will consider at least satisfactory.</p>
|
||||
<p>Before we start describing documentation types and formats, we should stress one important statement: developers interact with your help articles totally unlike you expect them to. Remember yourself working on the project: you make quite specific actions.</p>
|
||||
<ol>
|
||||
@ -3884,13 +3889,13 @@ ProgramContext.dispatch = (action) => {
|
||||
</ol>
|
||||
<p>In fact, newcomers (e.g. those developers who are not familiar with the API) usually want just one thing: to assemble the code that solves their problem out of existing code samples and never return to this issue again. Sounds not exactly reassuringly, given the amount of work invested into the API and its documentation development, but that's what the reality looks like. Also, that's the root cause of developers' dissatisfaction with the docs: it's literally impossible to have articles covering exactly that problem the developer comes with being detailed exactly to the extent the developer knows the API concepts. In addition, non-newcomers (e.g. those developers who have already learned the basics concepts and are now trying to solve some advanced problems) do not need these “mixed examples” articles as they look for some deeper understanding.</p>
|
||||
<h4>Introductory Notes</h4>
|
||||
<p>Documentation frequently suffers from being excessively clerical; it's being written using formal terminology (which often requires reading the glossary before the actual docs) and being unreasonably inflated. So instead of a two-word answer to the user's question a couple of paragraphs are conceived — a practice we strongly disapprove of. The perfect documentation must be simple and laconic, and all the terms must be either explained in the text or given a reference to such an explanation. However, “simple” doesn't mean “illiterate”: remember, the documentation is the face of your product, so grammar errors and improper usage of terms are unacceptable.</p>
|
||||
<p>Also, keep in mind that documentation will be used for searching as well, so every page should contain all the keywords required to be properly ranked by search engines. This requirement somehow contradicts the simple-and-laconic principle; that's the way.</p>
|
||||
<p>Documentation frequently suffers from being excessively clerical; it's being written using formal terminology (which often requires reading the glossary before the actual docs) and is frequently unreasonably inflated. So instead of a two-word answer to a user's question, a couple of paragraphs is conceived — a practice we strongly disapprove of. The perfect documentation must be simple and laconic, and all the terms must be either explained in the text or given a reference to such an explanation. However, “simple” doesn't mean “illiterate”: remember, the documentation is the face of your product, so grammar errors and improper usage of terms are unacceptable.</p>
|
||||
<p>Also, keep in mind that documentation will be used for searching as well, so every page should contain all the keywords required to be properly ranked by search engines. Unfortunately, this requirement contradicts the simple-and-laconic principle; that's the way.</p>
|
||||
<h4>Documentation Content Types</h4>
|
||||
<h5><a href="#chapter-30-paragraph-1" id="chapter-30-paragraph-1" class="anchor">1. Specification / Reference</a></h5>
|
||||
<p>Any documentation starts with a formal functional description. This content type is the most inconvenient to use, but you must provide it. A reference is the hygienic minimum of the API documentation. If you don't have a doc that describes all methods, parameters, options, variable types, and their allowed values, then it's not an API but amateur dramatics.</p>
|
||||
<p>Today, a reference must be also a machine-readable specification, e.g. comply with some standard, for example, OpenAPI.</p>
|
||||
<p>A specification must comprise not only formal descriptions but implicit agreements as well, such as the event generation order or unobvious side-effects of the API methods. Its most important applied value is advisory consulting: developers will refer to it to clarify unobvious situations.</p>
|
||||
<p>The specification must comprise not only formal descriptions but implicit agreements as well, such as the event generation order or unobvious side-effects of the API methods. Its important applied value is advisory consulting: developers will refer to it to clarify unobvious situations.</p>
|
||||
<p><strong>Importantly</strong>, formal specification <em>is not documentation</em> per se. The documentation is <em>the words you write</em> in the descriptions of each field and method. Without them, the specification might be used just for checking whether your namings are fine enough for developers to guess their meaning.</p>
|
||||
<p>Today, the method nomenclature descriptions are frequently additionally exposed as ready-to-use request collections or code fragments for Postman or analogous tools.</p>
|
||||
<h5><a href="#chapter-30-paragraph-2" id="chapter-30-paragraph-2" class="anchor">2. Code Samples</a></h5>
|
||||
@ -3916,23 +3921,22 @@ ProgramContext.dispatch = (action) => {
|
||||
<li>proper reactions to program errors that could happen;</li>
|
||||
<li>detailed studies on advanced API functionality (with detailed examples).</li>
|
||||
</ul>
|
||||
<p>As usual, a tutorial comprises a common section (basic terms and concepts, notation keys) and a set of sections regarding each functional domain exposed via the API.</p>
|
||||
<p>Usually, tutorials contain a “Quick Start” (“Hello, world!”) section: the smallest possible code sample that would allow developers to build a small app atop the API. “Quick Starts” aim to cover two needs:</p>
|
||||
<p>Usually, a tutorial comprises a common section (basic terms and concepts, notation keys) and a set of sections regarding each functional domain exposed via the API. Frequently, tutorials contain a “Quick Start” (“Hello, world!”) section: the smallest possible code sample that would allow developers to build a small app atop the API. “Quick Starts” aim to cover two needs:</p>
|
||||
<ul>
|
||||
<li>to provide a default entry-point, the easiest to understand and the most useful text for those who heard about your API for the first time;</li>
|
||||
<li>to engage developers, to make them touch the service by a mean of a real-world example.</li>
|
||||
</ul>
|
||||
<p>Also, “Quick starts” are a good indicator of how exactly well did you do your homework of identifying the most important use cases and providing helper methods. If your Quick Start comprises more than ten lines of code, you have definitely done something wrong.</p>
|
||||
<h5><a href="#chapter-30-paragraph-5" id="chapter-30-paragraph-5" class="anchor">5. Frequently Asked Questions and a Knowledge Base</a></h5>
|
||||
<h5><a href="#chapter-30-paragraph-5" id="chapter-30-paragraph-5" class="anchor">5. Frequently Asked Questions and Knowledge Bases</a></h5>
|
||||
<p>After you publish the API and start supporting users (see the previous chapter) you will also accumulate some knowledge of what questions are asked most frequently. If you can't easily integrate answers into the documentation, it's useful to compile a specific “Frequently Asked Questions” (aka FAQ) article. A FAQ article must meet the following criteria:</p>
|
||||
<ul>
|
||||
<li>address the real questions (you might frequently find FAQs that were reflecting not users' needs, but the API owner's desire to repeat some important information once more; it's useless, or worse — annoying; perfect examples of this anti-pattern realization might be found on any bank or air company website);</li>
|
||||
<li>both questions and answers must be formulated clearly and succinctly; it's acceptable (and even desirable) to provide links to corresponding reference and tutorial articles, but the answer itself can't be longer than a couple of paragraphs.</li>
|
||||
</ul>
|
||||
<p>Also, FAQs are a convenient place to explicitly highlight the advantages of the API. In a question-answer form, you might demonstrably show how your API solves complex problems easily and handsomely. (Or at least, solves them generally unlike the competitors' products.)</p>
|
||||
<p>Also, FAQs are a convenient place to explicitly highlight the advantages of the API. In a question-answer form, you might demonstrably show how your API solves complex problems easily and handsomely. (Or at least, <em>solves them</em>, unlike the competitors' products.)</p>
|
||||
<p>If technical support conversations are public, it makes sense to store all the questions and answers as a separate service to form a knowledge base, e.g. a set of “real-life” questions and answers.</p>
|
||||
<h5><a href="#chapter-30-paragraph-6" id="chapter-30-paragraph-6" class="anchor">6. Offline Documentation</a></h5>
|
||||
<p>Though we live in the online world, an offline version of the documentation (in a form of a generated doc file) still might be useful — first of all, as a snapshot of the API valid for a specific date.</p>
|
||||
<p>Though we live in the online world, an offline version of the documentation (in a form of a generated doc file) still might be useful — first of all, as a snapshot of the API specification valid for a specific date.</p>
|
||||
<h4>Content Duplication Problems</h4>
|
||||
<p>A significant problem that harms documentation clarity is API versioning: articles describing the same entity across different API versions are usually quite similar. Organizing convenient searching capability over such datasets is a problem for internal and external search engines as well. To tackle this problem ensure that:</p>
|
||||
<ul>
|
||||
@ -3953,7 +3957,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p><a href="https://forms.gle/WPdQ9KsJt3fxqpyw6">Yes / No</a></p><div class="page-break"></div><h3><a href="#api-product-testing" class="anchor" id="api-product-testing">Chapter 31. The Testing Environment</a><a href="#chapter-31" class="secondary-anchor" id="chapter-31"> </a></h3>
|
||||
<p>If the operations executed via the API imply consequences for end users or partners (cost money, in particular) you must provide a test version of the API. In this testing API, real-world actions either don't happen at all (for instance, orders are created but nobody serves them) or are simulated by cheaper means (let's say, instead of sending an SMS to a user, an email is sent to the developer's mailbox).</p>
|
||||
<p>However, in many cases having a test version is not enough — like in our coffee-machine API example. If an order is created but not served, partners are not able to test the functionality of delivering the order or requesting a refund. To run the full cycle of testing, developers need the capability of pushing the order through stages, as this would happen in reality.</p>
|
||||
<p>A direct solution to this problem is providing a full set of testing APIs and administrative interfaces. It means that developers will need to run a second application in parallel — the one you're giving to coffee shops so they might get and serve orders (and if there is a delivery functionality, the third app as well: the courier's one) — and make all these actions that coffee shop staff normally does. Obviously, that's not an ideal solution, because of several reasons:</p>
|
||||
<p>A direct solution to this problem is providing test versions for a full set of APIs and administrative interfaces. It means that developers will be able to run a second application in parallel — the one you're giving to coffee shops so they might get and serve orders (and if there is a delivery functionality, the third app as well: the courier's one) — and make all these actions that coffee shop staff normally does. Obviously, that's not an ideal solution, because of several reasons:</p>
|
||||
<ul>
|
||||
<li>developers of end user applications will need to additionally learn how coffee shop and courier apps work, which has nothing to do with the task they're solving;</li>
|
||||
<li>you will need to invent and implement some matching algorithm: an order made through a test application must be assigned to a specific virtual courier; this actually means creating an isolated virtual “sandbox” (meaning — a full set of services) for each specific partner;</li>
|
||||
@ -3965,25 +3969,24 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Ideally, you should provide helper methods for any actions that are conducted by people in the production environment. It makes sense to ship this meta-API complete with ready-to-use scripts or request collections that show the correct API call orders for standard scenarios.</p>
|
||||
<p>The disadvantage of this approach is that client developers still need to know how the “flip side” of the system works, though in simplified terms.</p>
|
||||
<h5><a href="#chapter-31-paragraph-2" id="chapter-31-paragraph-2" class="anchor">2. The Simulator of Pre-Defined Scenarios</a></h5>
|
||||
<p>The alternative to providing the testing environment API is simulating the working scenarios. In this case, the testing environment takes control over “underwater” parts of the system and “plays” all external agents' actions. In our coffee example, that means that, after the order is submitted, the system will simulate all the preparation steps and then the delivery of the beverage to the customer.</p>
|
||||
<p>The alternative to providing the testing environment API is simulating the working scenarios. In this case, the testing environment takes control over “underwater” parts of the system and “plays out” all external agents' actions. In our coffee example, that means that, after the order is submitted, the system will simulate all the preparation steps and then the delivery of the beverage to the customer.</p>
|
||||
<p>The advantage of this approach is that it demonstrates vividly how the system works according to the API vendor design plans, e.g. in which sequence the events are generated, and which stages the order passes through. It also reduces the chance of making mistakes in testing scripts, as the API vendor guarantees the actions will be executed in the correct order with the right parameters.</p>
|
||||
<p>The main disadvantage is the necessity to create a separate scenario for each unhappy path (effectively, for every possible error), and give developers the capability of denoting which scenario they want to run. (For example, like that: if there is a pre-agreed comment to the order, the system will simulate a specific error, and developers will be able to write and debug the code that deals with the error.)</p>
|
||||
<h4>The Automation of Testing</h4>
|
||||
<p>Your final goal in implementing testing APIs, regardless of which option you choose, is allowing partners to automate the QA process for their products. The testing environment should be developed with this purpose in mind; for example, if an end user might be brought to a 3-D Secure page to pay for the order, the testing environment API must provide some way of simulating the successful (or not) passing of this step. Also, in both variants, it's possible (and desirable) to allow running the scenarios in a fast-forward manner that will allow making auto-testing much faster than manual testing.</p>
|
||||
<p>Of course, not every partner will be able to employ this possibility (which also means that a “manual” way of testing usage scenarios must also be supported alongside the programmatical one) simply because not every business might afford to hire a QA automation engineer. Nevertheless, the ability to write such auto-tests is your API's huge competitive advantage from a technically advanced partner's point of view.</p><div class="page-break"></div><h3><a href="#api-product-expectations" class="anchor" id="api-product-expectations">Chapter 32. Managing expectations</a><a href="#chapter-32" class="secondary-anchor" id="chapter-32"> </a></h3>
|
||||
<p>Of course, not every partner will be able to employ this possibility (which also means that a “manual” way of testing usage scenarios must always be supported alongside the programmatical one) simply because not every business might afford to hire a QA automation engineer. Nevertheless, the ability to write such auto-tests is your API's huge competitive advantage from a technically advanced partner's point of view.</p><div class="page-break"></div><h3><a href="#api-product-expectations" class="anchor" id="api-product-expectations">Chapter 32. Managing expectations</a><a href="#chapter-32" class="secondary-anchor" id="chapter-32"> </a></h3>
|
||||
<p>Finally, the last aspect we would like to shed the light on is managing partners' expectations regarding the further development of the API. If we talk about consumer qualities, APIs differ little from other B2B software products: in both cases, you need to form some understanding of SLA conditions, available features, interface responsiveness and other characteristics that are important for clients. Still, APIs have their specificities</p>
|
||||
<h4>Versioning and Application Lifecycle</h4>
|
||||
<p>Ideally, the API once published should live eternally; but as we all are reasonable people, we do understand it's impossible in the real life. Even if we continue supporting older versions, they will still become outdated eventually, and partners will need to rewrite the code to use newer functionality.</p>
|
||||
<p>The author of this book formulates the rule of issuing new major API versions like this: the period of time after which partners will need to rewrite the code should coincide with the application lifespan in the subject area. Any program will once become obsolete and will be rewritten; and if during this re-developing partners need to also switch to a newer API version, it will be met with some degree of understanding. Of course, in different subject areas, this timespan differs, depending on the evolution rate of the underlying platform.</p>
|
||||
<p>Apart from updating <em>major</em> versions, sooner or later you will face issues with accessing some outdated <em>minor</em> versions as well. As we mentioned in the <a href="#back-compat-iceberg-waterline">“On the Waterline of the Iceberg”</a> chapter, even fixing bugs might eventually lead to breaking some integrations, and that naturally leads us to the necessity of keeping older <em>minor</em> versions of the API until the partner resolves the problem.</p>
|
||||
<p>In this aspect, integrating with large companies that have a dedicated software engineering department differs dramatically from providing a solution to individual amateur programmers: from one side, the former are much more likely to find undocumented features and unfixed bugs in your code; on the other side, because of the internal bureaucracy, fixing the related issues might easily take months, save not years. The common recommendation there is to maintain old minor API versions for a period of time long enough for the most dilatory partner to switch no the newest version.</p>
|
||||
<p>The author of this book formulates the rule of issuing new major API versions like this: the period of time after which partners will need to rewrite the code should coincide with the application lifespan in the subject area (see <a href="#back-compat-statement">“The Backwards Compatibility Problem Statement”</a> chapter). Apart from updating <em>major</em> versions, sooner or later you will face issues with accessing some outdated <em>minor</em> versions as well. As we mentioned in the <a href="#back-compat-iceberg-waterline">“On the Waterline of the Iceberg”</a> chapter, even fixing bugs might eventually lead to breaking some integrations, and that naturally leads us to the necessity of keeping older <em>minor</em> versions of the API until the partner resolves the problem.</p>
|
||||
<p>In this aspect, integrating with large companies that have a dedicated software engineering department differs dramatically from providing a solution to individual amateur programmers: on one hand, the former are much more likely to find undocumented features and unfixed bugs in your code; on the other hand, because of the internal bureaucracy, fixing the related issues might easily take months, save not years. The common recommendation there is to maintain old minor API versions for a period of time long enough for the most dilatory partner to switch no the newest version.</p>
|
||||
<h4>Supporting Platforms</h4>
|
||||
<p>Another aspect crucial to interacting with large integrators is supporting a zoo of platforms (browsers, programming languages, protocols, operating systems) and their versions. As usual, big companies have their own policies on which platforms they support, and these policies might sometimes contradict common sense. (Let's say, it's rather a time to abandon TLS 1.2, but many integrators continue working through this protocol, or even the earlier ones.)</p>
|
||||
<p>Formally speaking, ceasing support of a platform <em>is</em> a backwards-incompatible change, and might lead to breaking some integration for some end users. So it's highly important to have clearly formulated policies on which platforms are supported based on which criteria. In the case of mass public APIs, that's usually simple (like, API vendor promises to support platforms that have more than N% penetration, or, even easier, just last M versions of a platform); in the case of commercial APIs, it's always a bargain based on the estimations, how much will non-supporting a specific platform would cost to a company. And of course, the outcome of the bargain must be stated in the contracts — what exactly you're promising to support during which period of time.</p>
|
||||
<p>Formally speaking, ceasing support of a platform <em>is</em> a backwards-incompatible change, and might lead to breaking some integration for some end users. So it's highly important to have clearly formulated policies regarding which platforms are supported based on which criteria. In the case of mass public APIs, that's usually simple (like, API vendor promises to support platforms that have more than N% penetration, or, even easier, just last M versions of a platform); in the case of commercial APIs, it's always a bargain based on the estimations, how much will non-supporting a specific platform would cost to a company. And of course, the outcome of the bargain must be stated in the contracts — what exactly you're promising to support during which period of time.</p>
|
||||
<h4>Moving Forward</h4>
|
||||
<p>Finally, apart from those specific issues, your customers must be caring about more general questions: could they trust you? Could they rely on your API evolving, absorbing modern trends, or will they eventually find the integration with your API on the scrapyard of history? Let's be honest: given all the uncertainties of the API product vision, we are very much interested in the answers as well. Even the Roman viaduct, though remaining backwards-compatible for two thousand years, has been a very archaic and non-reliable way of solving customers' problems for quite a long time.</p>
|
||||
<p>You might work with these customer expectations by publishing roadmaps. It's quite common that many companies avoid publicly announcing their concrete plans (for a reason, of course). Nevertheless, in the case of APIs, we strongly recommend providing the roadmaps, even if they are tentative and lack precise dates — <em>especially</em> if we talk about deprecating some functionality. Announcing these promises (given the company keeps them, of course) is a very important competitive advantage to every kind of consumer.</p>
|
||||
<p>With this, we would like to conclude this book. We hope that the principles and the concepts we have outlined will help you in creating APIs that fit all the developer', business', and end user's needs, and in expanding them (while maintaining the backwards compatibility) for the next two thousand years (or maybe more).</p><div class="page-break"></div></section></article><script>const activeElements = [
|
||||
<p>With this, we would like to conclude this book. We hope that the principles and the concepts we have outlined will help you in creating APIs that fit all the developers, businesses, and end users' needs, and in expanding them (while maintaining the backwards compatibility) for the next two thousand years or so.</p><div class="page-break"></div></section></article><script>const activeElements = [
|
||||
'.side-panel .close',
|
||||
'nav.page-main',
|
||||
'.side-panel .copy-button',
|
||||
|
BIN
docs/API.en.pdf
BIN
docs/API.en.pdf
Binary file not shown.
BIN
docs/API.ru.epub
BIN
docs/API.ru.epub
Binary file not shown.
@ -3735,13 +3735,13 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Другая опасность заключается в том, что ключ могут банально украсть у добросовестного партнёра; в случае клиентских и веб-приложений это довольно тривиально.</p>
|
||||
<p>Может показаться, что в случае предоставления серверных API проблема воровства ключей неактуальна, но, на самом деле, это не так. Предположим, что партнёр предоставляет свой собственный публичный сервис, который «под капотом» использует ваше API. Это часто означает, что в сервисе партнёра есть эндпойнт, предназначенный для конечных пользователей, который внутри делает запрос к API и возвращает результат, и этот эндпойнт может использоваться злоумышленником как эквивалент API. Конечно, можно объявить такой фрод проблемой партнёра, однако было бы, во-первых, наивно ожидать от каждого партнёра реализации собственной антифрод-системы, которая позволит выявлять таких недобросовестных пользователей, и, во-вторых, это попросту неэффективно: очевидно, что централизованная система борьбы с фродерами всегда будет более эффективной, нежели множество частных любительских реализаций. К томе же, и серверные ключи могут быть украдены: это сложее, чем украсть клиентские, но не невозможно. Популярный API рано или поздно столкнётся с тем, что украденные ключи будут выложены в свободный доступ (или владелец ключа просто будет делиться им со знакомыми по доброте душевной).</p>
|
||||
<p>Так или иначе, встаёт вопрос независимой валидации: каким образом можно проконтролировать, действительно ли API используется потребителем в соответствии с пользовательским соглашением.</p>
|
||||
<p>Мобильные приложения удобно отслеживаются по идентификатору приложения в соответствующем сторе (Google Play, App Store и другие), поэтому разумно требовать от партнёров идентифицировать приложение при подключении API. Вебсайты с некоторой точностью можно идентифицировать по заголовкам Referer или Origin (и для надёжности можно так же потребовать от партнёра указывать домен сайта при инициализации API).</p>
|
||||
<p>Мобильные приложения удобно отслеживаются по идентификатору приложения в соответствующем сторе (Google Play, App Store и другие), поэтому разумно требовать от партнёров идентифицировать приложение при подключении API. Вебсайты с некоторой точностью можно идентифицировать по заголовкам <code>Referer</code> или <code>Origin</code> (и для надёжности можно так же потребовать от партнёра указывать домен сайта при инициализации API).</p>
|
||||
<p>Эти данные сами по себе не являются надёжными; важно то, что они позволяют проводить кросс-проверки:</p>
|
||||
<ul>
|
||||
<li>если ключ был выпущен для одного домена, но запросы приходят с Referer-ом другого домена — это повод разобраться в ситуации и, возможно, забанить возможность обращаться к API с этим Referer-ом или этим ключом;</li>
|
||||
<li>если ключ был выпущен для одного домена, но запросы приходят с <code>Referer</code>-ом другого домена — это повод разобраться в ситуации и, возможно, забанить возможность обращаться к API с этим <code>Referer</code>-ом или этим ключом;</li>
|
||||
<li>если одно приложение инициализирует API с указанием ключа другого приложения — это повод обратиться к администрации стора с требованием удалить одно из приложений.</li>
|
||||
</ul>
|
||||
<p><strong>NB</strong>: не забудьте разрешить безлимитное использование с Referer-ом <code>localhost</code> и <code>127.0.0.1</code> / <code>[::1]</code>, а также из вашей собственной песочницы, если она есть. Да, в какой-то момент злоумышленники поймут, что на такие Referer-ы не действуют ограничения, но это точно произойдёт гораздо позже, чем вы по неосторожности забаните локальную разработку или собственный сайт документации.</p>
|
||||
<p><strong>NB</strong>: не забудьте разрешить безлимитное использование с <code>Referer</code>-ом <code>localhost</code> и <code>127.0.0.1</code> / <code>[::1]</code>, а также из вашей собственной песочницы, если она есть. Да, в какой-то момент злоумышленники поймут, что на такие <code>Referer</code>-ы не действуют ограничения, но это точно произойдёт гораздо позже, чем вы по неосторожности забаните локальную разработку или собственный сайт документации.</p>
|
||||
<p>Общий вывод из вышеизложенного таков:</p>
|
||||
<ul>
|
||||
<li>очень желательно иметь формальную идентификацию пользователей (API-ключи как самая распространённая практика, либо указание контактных данных, таких как домен вебсайта или идентификатор приложения в сторе, при инициализации API);</li>
|
||||
@ -3751,8 +3751,8 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Если к партнёрам вы можете предъявлять какие-то требования по самоидентификации, то от конечных пользователей требовать раскрытия информации о себе в большинстве случаев не представляется возможным. Все методы контроля, описанные ниже, являются неточными и зачастую эвристическими. (Даже если функциональность партнёрских приложений предоставляется только после регистрации пользователя и вы имеете к этой регистрации доступ, вы всё ещё гадаете, т.к. аккаунт это не то же самое, что и отдельный пользователь: несколько различных людей могут пользоваться одним профилем или, наоборот, у одного человека может быть множество профилей.) Кроме того, следует иметь в виду, что сбор подобного рода информации может регулироваться законодательно (хотя большей частью речь пойдёт об анонимизированных данных, но и они могут быть регламентированы).</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>Самый простой и очевидный показатель — это ip-адреса; их невозможно подделать (в том смысле, что сервер API всегда знает адрес вызывающего клиента), и поэтому статистика по уникальным ip довольно показательна.</p>
|
||||
<p>Если API предоставляется как server-to-server сервис, доступа к IP-адресу конечного пользователя может и не быть, однако весьма разумно в такой ситуации требовать от партнёра пробрасывать IP-адрес клиента (например, в виде заголовка X-Forwarded-For) — в том числе для того, чтобы помочь партнёрам бороться с фродом и неправомерным использованием API.</p>
|
||||
<p>Самый простой и очевидный показатель — это IP-адреса; их невозможно подделать (в том смысле, что сервер API всегда знает адрес вызывающего клиента), и поэтому статистика по уникальным IP довольно показательна.</p>
|
||||
<p>Если API предоставляется как server-to-server сервис, доступа к IP-адресу конечного пользователя может и не быть, однако весьма разумно в такой ситуации требовать от партнёра пробрасывать IP-адрес клиента (например, в виде заголовка <code>X-Forwarded-For</code>) — в том числе для того, чтобы помочь партнёрам бороться с фродом и неправомерным использованием API.</p>
|
||||
<p>До недавнего времени ip-адрес как единица подсчёта статистики был ещё и удобен тем, что обзавестись большим пулом уникальных адресов было достаточно дорого. Однако с распространением ipv6 это ограничение перестало быть актуальным; скорее, ipv6 ярко подсветил тот факт, что не стоит ограничиваться только подсчётом уникальных ip. Необходимо следить за несколькими агрегатами:</p>
|
||||
<ul>
|
||||
<li>суммировать статистику по подсетям, т.е. вести иерархические подсчёты (количество уникальных сетей /8, /16, /24 и так далее);</li>
|
||||
@ -3776,7 +3776,7 @@ ProgramContext.dispatch = (action) => {
|
||||
</ul>
|
||||
<h5><a href="#chapter-28-paragraph-1" id="chapter-28-paragraph-1" class="anchor">1. Идентификация подозрительных пользователей</a></h5>
|
||||
<p>По большому счёту, здесь есть всего два подхода, которые мы можем применить — статический и динамический (поведенческий).</p>
|
||||
<p><em>Статически</em> мы отслеживаем подозрительную концентрацию активности (как описано в предыдущей главе), отмечая нехарактерно высокое количество запросов из определённых подсетей или Referer-ов (на самом деле, нам подойдёт <em>любая</em> информация, которая как-то делит пользователей на более-менее независимые группы — такая как, например, версия ОС или язык системы, если такие данные нам доступны).</p>
|
||||
<p><em>Статически</em> мы отслеживаем подозрительную концентрацию активности (как описано в предыдущей главе), отмечая нехарактерно высокое количество запросов из определённых подсетей или <code>Referer</code>-ов (на самом деле, нам подойдёт <em>любая</em> информация, которая как-то делит пользователей на более-менее независимые группы — такая как, например, версия ОС или язык системы, если такие данные нам доступны).</p>
|
||||
<p>При <em>поведенческом</em> анализе мы анализируем историю запросов одного конкретного пользователя и отмечаем нетипичное поведение — «нечеловеческий» порядок обхода эндпойнтов, слишком быстрый их перебор, etc.</p>
|
||||
<p><strong>Важно</strong>: когда мы здесь говорим о «пользователе», нам почти всегда придётся дублировать анализ для работы по ip-адресу, поскольку злоумышленник вовсе не обязан будет сохранять cookie или другой идентификационный токен, или будет ротировать набор таких токенов, чтобы затруднить идентификацию.</p>
|
||||
<h5><a href="#chapter-28-paragraph-2" id="chapter-28-paragraph-2" class="anchor">2. Запрос дополнительного фактора аутентификации</a></h5>
|
||||
@ -3785,7 +3785,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p><strong>NB</strong>. Вместо капчи здесь могут быть любые другие действия, вводящие дополнительные факторы аутентификации. Это может быть, например, подтверждение номера телефона или второй шаг протокола 3D-Secure. Важно здесь то, что запрос второго шага аутентификации должен быть предусмотрен в API, поскольку добавить его его обратно совместимым образом к существующим endpoint-ам нельзя.</p>
|
||||
<p>Другие популярные способы распознать робота — предложить ему приманку (honeypot) или использовать методы проверки среды исполнения (начиная от достаточно простых вроде исполнения JavaScript на странице и заканчивая технологиями проверки целостности приложения).</p>
|
||||
<h5><a href="#chapter-28-paragraph-3" id="chapter-28-paragraph-3" class="anchor">3. Ограничение доступа</a></h5>
|
||||
<p>Видимость богатства способов технической идентификации пользователей, увы, разбивается о суровую реальность наличия у вас очень скромных средств ограничения доступа. Бан по cookie / Referer-у / User-Agent-у практически не работает по той причине, что эти данные передаёт клиент, и он же легко может их подменить. По большому счёту, способов ограничения доступа у вас четыре:</p>
|
||||
<p>Видимость богатства способов технической идентификации пользователей, увы, разбивается о суровую реальность наличия у вас очень скромных средств ограничения доступа. Бан по cookie / <code>Referer</code>-у / <code>User-Agent</code>-у практически не работает по той причине, что эти данные передаёт клиент, и он же легко может их подменить. По большому счёту, способов ограничения доступа у вас четыре:</p>
|
||||
<ul>
|
||||
<li>бан пользователя по ip (подсети, автономной системе);</li>
|
||||
<li>требование обязательной идентификации пользователя (возможно, прогрессивной: логин в системе / логин с подтверждённым номером телефона / логин с подтверждением личности / логин с подтверждением личности и биометрией);</li>
|
||||
@ -3803,7 +3803,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Рассмотрим теперь второй вариант несанкционированного использования, когда злоумышленник крадёт API-ключ добросовестного партнёра и вставляет его в своё приложение. Запросы при этом генерируются настоящими пользователями, а значит капча никак не поможет — но помогут другие методы.</p>
|
||||
<ol>
|
||||
<li>
|
||||
<p>Ведение статистики по ip-адресам и сетям может помочь и здесь. Если приложение злоумышленника всё-таки не обычное приложение для честных потребителей, а какой-то закрытый сервис для ограниченного круга пользователей, этот факт будет виден на приборах (а если повезёт — то вы увидите ещё и подозрительные Referer-ы, закрытые для внешнего доступа).</p>
|
||||
<p>Ведение статистики по ip-адресам и сетям может помочь и здесь. Если приложение злоумышленника всё-таки не обычное приложение для честных потребителей, а какой-то закрытый сервис для ограниченного круга пользователей, этот факт будет виден на приборах (а если повезёт — то вы увидите ещё и подозрительные <code>Referer</code>-ы, закрытые для внешнего доступа).</p>
|
||||
</li>
|
||||
<li>
|
||||
<p>Предоставление возможности партнёрам ограничивать функциональность, которая доступна по ключу:</p>
|
||||
@ -3991,8 +3991,7 @@ ProgramContext.dispatch = (action) => {
|
||||
<p>Наконец, последний аспект, который хотелось бы осветить в рамках данного раздела — это управление ожиданиями партнёров в отношении развития вашего API. С точки зрения коммуникации потребительских качеств API мало отличается от любого другого B2B программного обеспечения: и там, и там вам нужно как-то сформировать у разработчиков и бизнеса понимание о допустимом SLA, объёме функциональности, отзывчивости интерфейсов и прочих пользовательских характеристиках. Однако у API как продукта есть и специфические особенности.</p>
|
||||
<h4>Версионирование и жизненный цикл приложений</h4>
|
||||
<p>Конечно, в идеальном случае однажды выпущенный API должен жить вечно; но, как разумные люди, мы понимаем, что в реальной жизни это невозможно. Даже если мы продолжаем поддерживать старые версии, они все равно морально устаревают: партнёры должны потратить ресурсы на переписывание кода под новые версии API, если хотят получить доступ к новой функциональности.</p>
|
||||
<p>Автор этой книги формулирует для себя золотое правило выпуска новых версий API так: период времени, после которого партнеру потребуется переписать свой код, должен совпадать с жизненным циклом приложений в вашей предметной области. Любая программа рано или поздно устареет и будет переписана; если в рамках этого переписывания будет осуществлён и переход на свежую версию API, партнёр, скорее всего, отнесётся к этому с пониманием. Разумеется, в разных областях этот интервал различен, и зависит от скорости эволюции самой нижележащей платформы.</p>
|
||||
<p>Помимо переключения <em>мажорных</em> версий API, рано или поздно встанет вопрос и о доступе к <em>минорным</em> версиям. Как мы упоминали в главе <a href="#back-compat-iceberg-waterline">«О ватерлинии айсберга»</a>, даже исправление ошибок в коде может привести к неработоспособности какой-то интеграции. Соответственно, может потребоваться и сохранение возможности зафиксировать <em>минорную</em> версию API до момента обновления затронутого кода партнёром.</p>
|
||||
<p>Автор этой книги формулирует для себя золотое правило выпуска новых версий API так: период времени, после которого партнеру потребуется переписать свой код, должен совпадать с жизненным циклом приложений в вашей предметной области (см. главу <a href="#back-compat-statement">«Постановка проблемы обратной совместимости»</a>). Помимо переключения <em>мажорных</em> версий API, рано или поздно встанет вопрос и о доступе к <em>минорным</em> версиям. Как мы упоминали в главе <a href="#back-compat-iceberg-waterline">«О ватерлинии айсберга»</a>, даже исправление ошибок в коде может привести к неработоспособности какой-то интеграции. Соответственно, может потребоваться и сохранение возможности зафиксировать <em>минорную</em> версию API до момента обновления затронутого кода партнёром.</p>
|
||||
<p>В этом аспекте интеграция с крупными компаниями, имеющими собственный отдел разработки, существенно отличается от взаимодействия с одиночными разработчиками-любителями: первые, с одной стороны, с гораздо большей вероятностью найдут в вашем API недокументированные возможности и неисправленные ошибки; с другой стороны, в силу большей бюрократичности внутренних процессов, исправление проблем может затянуться на месяцы, а то и годы. Общая рекомендация здесь — поддерживать возможность подключения старых минорных версий API достаточно долго для того, чтобы самый забюрократизированный партнёр успел переключиться на новую версию.</p>
|
||||
<h4>Поддержка платформ</h4>
|
||||
<p>Ещё один аспект, критически важный во взаимодействии с крупными интеграторами — это поддержка зоопарка платформ (браузеров, языков программирования, протоколов, операционных систем) и их версий. Как правило, большие компании имеют свои собственные стандарты, какие платформы они поддерживают, и эти стандарты могут входить в заметное противоречие со здравым смыслом. (Например, от TLS 1.2 в настоящий момент уже желательно отказываться, однако многие интеграторы продолжают работать именно через этот протокол, и это ещё в лучшем случае.)</p>
|
||||
|
BIN
docs/API.ru.pdf
BIN
docs/API.ru.pdf
Binary file not shown.
@ -1204,7 +1204,7 @@ One important implication: **never use increasing numbers as external identifier
|
||||
|
||||
With the API popularity growth, it will inevitably become necessary to introduce technical means of preventing illicit API usage, such as displaying captchas, setting honeypots, raising the “too many requests” exceptions, installing anti-DDoS proxies, etc. All these things cannot be done if the corresponding errors and messages were not described in the docs from the very beginning.
|
||||
|
||||
You are not obliged to actually generate those exceptions, but you might stipulate this possibility in the terms of service. For example, you might describe the `429 Too Many Requests` error or captcha redirect but implement the functionality when it's actually needed.
|
||||
You are not obliged to actually generate those exceptions, but you might stipulate this possibility in the Terms of Service (ToS). For example, you might describe the `429 Too Many Requests` error or captcha redirect but implement the functionality when it's actually needed.
|
||||
|
||||
It is extremely important to leave room for multi-factored authentication (such as TOTP, SMS, or 3D-secure-like technologies) if it's possible to make payments through the API. In this case, it's a must-have from the very beginning.
|
||||
|
||||
|
@ -4,15 +4,17 @@ As we described in the previous chapters, there are many API monetization models
|
||||
|
||||
Of course, the most explicit metric is money: if your API is monetized directly or attracts visitors to a monetized service, the rest of the chapter will be of little interest to you, maybe just as a case study. If, however, the contribution of the API to the company's income cannot be simply measured, you have to stick to other, synthetic, indicators.
|
||||
|
||||
The obvious key performance indicator (KPI) \#1 is the number of end users and the number of integrations (i.e. partners using the API). Normally, they are in some sense a business health barometer: if there is a normal competitive situation among the API suppliers, and all of them are more or less in the same position, then the figure of how many developers (and consequently, how many end users) are using the API is the main metric of the success of the API product.
|
||||
The obvious key performance indicator (KPI) \#1 is the number of end users and the number of integrations (i.e. partners using the API). Normally, they are in some sense a business health barometer: if there is a normal competitive situation among the API suppliers, and all of them are more or less in the same position, then the figure of how many developers (and consequently, how many end users) are using the API is the main metric of success of the API product.
|
||||
|
||||
However, sheer numbers might be deceiving, especially if we talk about free-to-use integrations. There are several factors that twist the statistics:
|
||||
However, sheer numbers might be deceiving, especially if we talk about free-to-use integrations. There are several factors that make them less reliable:
|
||||
|
||||
* the high-level API services that are meant for straightforward integration (see the previous chapter) are significantly distorting the statistics if the competitors don't provide such services; typically, for one full-scale integration there will be tens, maybe hundreds, of those lightweight embedded widgets; thereby, it's crucial to have partners counted for each kind of the integration independently;
|
||||
* the high-level API services that are meant for point-and-click integration (see the previous chapter) are significantly distorting the statistics, especially if the competitors don't provide such services; typically, for one full-scale integration there will be tens, maybe hundreds, of those lightweight embedded widgets;
|
||||
|
||||
* thereby, it's crucial to have partners counted for each kind of the integration independently;
|
||||
|
||||
* partners tend to use the API in suboptimal ways:
|
||||
|
||||
* embed it at every site page / application screen instead of only those where end users can really interact with the API;
|
||||
* embed it at every website page / application screen instead of only those where end users can really interact with the API;
|
||||
* put widgets somewhere deep in the page / screen footer, or hide it behind spoilers;
|
||||
* initialize a broad range of API modules, but use only a limited subset of them;
|
||||
|
||||
@ -35,13 +37,13 @@ Let us summarize the paragraph:
|
||||
|
||||
#### SLA
|
||||
|
||||
This chapter would be incomplete if we didn't mention the “hygienic” KPI — the service level and the service availability. We won't be describing the concept in detail, as the API SLA isn't any different from any other digital services SLAs. Let us point out that this metric must be tracked, especially if we talk about pay-to-use APIs. However, in many cases, API vendors prefer to offer rather loose SLAs, treating the provided functionality as data access or a content licensing service.
|
||||
This chapter would be incomplete if we didn't mention the “hygienic” KPI — the service level and the service availability. We won't be describing the concept in detail, as the API SLA isn't any different from any other digital services SLAs. Let us just state that this metric must be tracked, especially if we talk about pay-to-use APIs. However, in many cases, API vendors prefer to offer rather loose SLAs, treating the provided functionality as a data access or content licensing service.
|
||||
|
||||
Still, let us re-iterate once more: any problems with your API are automatically multiplied by the number of partners you have, especially if the API is vital for them, e.g. the API outage makes the main functionality of their services unavailable. (And actually, because of the above-mentioned reasons, the average quality of integrations implies that partners' services will suffer even if the availability of the API is not formally speaking critical for them, but because developers use it excessively and do not bother with proper error handling.)
|
||||
|
||||
It is important to mention that predicting the workload for the API service is rather complicated. Sub-optimal API usage, e.g. initializing the API in those application and website parts where it's not actually needed, might lead to a colossal increase in the number of requests after changing a single line of partner's code. The safety margin for an API service must be much higher than for a regular service for end users — it must survive the situation of the largest partner suddenly starting querying the API on every page and every application screen. (If the partner is already doing that, then the API must survive doubling the load: imagine the partner accidentally starts initializing the API twice on each page / screen.)
|
||||
It is important to mention that predicting the workload for the API service is rather complicated. Sub-optimal API usage, e.g. initializing the API in those application and website parts where it's not actually needed, might lead to a colossal increase in the number of requests after changing a single line of partner's code. The safety margin for an API service must be much higher than for a regular service for end users — it must survive the situation of the largest partner suddenly starting querying the API on every page and every application screen. (If the partner is already doing that, then the API must survive doubling the load if the partner by accident starts initializing the API twice on each page / screen.)
|
||||
|
||||
Another extremely important hygienic minimum is the informational security of the API service. In the worst-case scenario, namely, if an API service vulnerability allows for exploiting partner applications, one security loophole will in fact be exposed *in every partner application*. Needless to say that the cost of such a mistake might be overwhelmingly colossal, even if the API itself is rather trivial and has no access to sensitive data (especially if we talk about webpages where no “sandbox” for third-party scripts exists, and any piece of code might let's say track the data entered in forms). API services must provide the maximum protection level (for example, choose cryptographical protocols with a certain overhead) and promptly react to any messages regarding possible vulnerabilities.
|
||||
Another extremely important hygienic minimum is the informational security of the API service. In the worst-case scenario, namely, if an API service vulnerability allows for exploiting partner applications, one security loophole will in fact be exposed *in every partner application*. Needless to say that the cost of such a mistake might be overwhelmingly colossal, even if the API itself is rather trivial and has no access to sensitive data (especially if we talk about webpages where no “sandbox” for third-party scripts exists, and any piece of code might let's say track the data entered in forms). API services must provide the maximum protection level (for example, choose cryptographical protocols with a certain overhead) and promptly react to any reports regarding possible vulnerabilities.
|
||||
|
||||
#### Comparing to Competitors
|
||||
|
||||
@ -50,4 +52,4 @@ While measuring KPIs of any service, it's important not only to evaluate your ow
|
||||
* is your service growing faster than the market itself or is the rate the same, or is it even less?
|
||||
* what proportion of the growth is caused by the growth of the market, and what is related to your efforts?
|
||||
|
||||
Getting answers to those questions might be quite non-trivial in the case of API services. Indeed, how could you learn how many integrations has your competitor had during the same period of time, and what number of target actions had happened on their platform? Sometimes, the providers of popular analytical tools might help you with this, but usually, you have to monitor the potential partners' apps and websites and gather the statistics regarding APIs they're using. The same applies to market research: unless your niche is significant enough for some analytical company to conduct market research, you will have to either commission such a study or make your own estimations — conversely, through interviewing potential customers.
|
||||
Getting answers to those questions might be quite non-trivial in the case of API services. Indeed, how could you learn how many integrations has your competitor had during the same period of time, and what number of target actions had happened on their platform? Sometimes, the providers of popular analytical tools might help you with this, but usually, you have to monitor the potential partners' apps and websites and gather the statistics regarding APIs they're using. The same applies to market research: unless your niche is significant enough for some analytical company to conduct a study, you will have to either commission such work or make your own estimations — conversely, through interviewing potential customers.
|
@ -5,7 +5,7 @@ In the context of working with an API, we talk about two kinds of users of the s
|
||||
* users-developers, e.g. your partners writing code atop of the API;
|
||||
* end users interacting with applications implemented by the users-developers.
|
||||
|
||||
In most cases, you need to have both of them identified (in a technical sense: discern one unique visitor from another) to have answers to the following questions:
|
||||
In most cases, you need to have both of them identified (in a technical sense: discern one unique customer from another) to have answers to the following questions:
|
||||
|
||||
* how many users are interacting with the system (simultaneously, daily, monthly, and yearly)?
|
||||
* how many actions does each user make?
|
||||
@ -23,7 +23,7 @@ In the case of commercial APIs, the quality and timeliness of gathering this dat
|
||||
|
||||
Let's start with the first user category, e.g. API business partners-developers. The important remark: there are two different entities we must learn to identify, namely applications and their owners.
|
||||
|
||||
An application is roughly speaking a logically separate case of API usage, usually — literally an application (mobile or desktop one) or a website, e.g. some technical entity. Meanwhile, an owner is a legal body that you have the API usage agreement signed with, e.g. a legal body. if API tariffs imply some limits and/or tariffs depend on the type of the service or the way it uses the API, this automatically means the necessity to track one owner's applications separately.
|
||||
An application is roughly speaking a logically separate case of API usage, usually — literally an application (mobile or desktop one) or a website, e.g. some technical entity. Meanwhile, an owner is a legal body that you have the API usage agreement signed. If API Terms of Service (ToS) imply different limits and/or tariffs depending on the type of the service or the way it uses the API, this automatically means the necessity to track one owner's applications separately.
|
||||
|
||||
In the modern world, the factual standard for identifying both entities is using API keys: a developer who wants to start using an API must obtain an API key bound to their contact info. Thus the key identifies the application while the contact data identifies the owner.
|
||||
|
||||
@ -35,19 +35,19 @@ The main disadvantage of using API keys is that they *don't* allow for reliably
|
||||
|
||||
If there are free limits to API usage, there is a temptation to obtain many API keys bound to different owners to fit those free limits. You may raise the bar of having such multi-accounts by requiring, let's say, providing a phone number or bank card data, but there are popular services for automatically issuing both. Paying for a virtual SIM or credit card (to say nothing about buying the stolen ones) will always be cheaper than paying the proper API tariff — unless it's the API for creating those cards. Therefore, API key-based user identification (if you're not requiring the physical contract to be signed) does not mean you don't need to double-check whether users comply with the terms of service and do not issue several keys for one app.
|
||||
|
||||
Another problem is that an API key might be simply stolen from a lawful partner; in the case of client or web applications, that's quite trivial.
|
||||
Another problem is that an API key might be simply stolen from a lawful partner; in the case of mobile or web applications, that's quite trivial.
|
||||
|
||||
It might look like the problem is not that important in the case of server-to-server integrations, but it actually is. Imagine that a partner provides a public service of their own that uses your API under the hood. That usually means there is an endpoint in the partner's backend that performs a request to the API and returns the result, and this endpoint perfectly suits as a free replacement of the API access to a cybercriminal. Of course, you might say this fraud is a problem of partners', but, first, it would be naïve to expect each partner develops their own anti-fraud system, and, second, it's just sub-optimal: obviously, a centralized anti-fraud system would be way more effective than a bunch of amateur implementations. Also, server keys might also be stolen: it's much harder than stealing client keys but doable. With any popular API, sooner or later you will face the situation of stolen keys made available to the public (or a key owner just shares it with acquaintances out of the kindness of their heart).
|
||||
It might look like the problem is not that important in the case of server-to-server integrations, but it actually is. Imagine that a partner provides a public service of their own that uses your API under the hood. That usually means there is an endpoint in the partner's backend that performs a request to the API and returns the result, and this endpoint perfectly suits as a free replacement of direct access to the API for a cybercriminal. Of course, you might say this fraud is a problem of partners', but, first, it would be naïve to expect each partner develops their own anti-fraud system, and, second, it's just sub-optimal: obviously, a centralized anti-fraud system would be way more effective than a bunch of amateur implementations. Also, server keys might also be stolen: it's much harder than stealing client keys but doable. With any popular API, sooner or later you will face the situation of stolen keys made available to the public (or a key owner just shared it with acquaintances out of the kindness of their heart).
|
||||
|
||||
One way or another, a problem of independent validation arises: how can we control whether the API endpoint is requested by a user in compliance with the terms of service?
|
||||
|
||||
Mobile applications might be conveniently tracked through their identifiers in the corresponding store (Google Play, App Store, etc.), so it makes sense to require this identifier to be passed by partners as an API initialization parameter. Websites with some degree of confidence might be identified by the Referer and Origin HTTP headers.
|
||||
Mobile applications might be conveniently tracked through their identifiers in the corresponding store (Google Play, App Store, etc.), so it makes sense to require this identifier to be passed by partners as an API initialization parameter. Websites with some degree of confidence might be identified by the `Referer` and `Origin` HTTP headers.
|
||||
|
||||
This data is not itself reliable, but it allows for making cross-checks:
|
||||
* if the key was issued for one specific domain but requests are coming with a different Referer, it makes sense to investigate the situation and maybe ban the possibility to access the API with this Referer or this key;
|
||||
* if an application initializes API by providing the key registered to another application, it makes sense to contact the store administration and ask for removing one of the apps.
|
||||
* if a key was issued for one specific domain but requests are coming with a different `Referer`, it makes sense to investigate the situation and maybe ban the possibility to access the API with this `Referer` or this key;
|
||||
* if an application initializes API by providing a key registered to another application, it makes sense to contact the store administration and ask for removing one of the apps.
|
||||
|
||||
**NB**: don't forget to set infinite limits for using the API with the `localhost`, `127.0.0.1` / `[::1]` Referers, and also for your own sandbox if it exists. Yes, abusers will sooner or later learn this fact and will start using it, but otherwise, you will ban local development and your own website much sooner than that.
|
||||
**NB**: don't forget to set infinite limits for using the API with the `localhost`, `127.0.0.1` / `[::1]` `Referer`s, and also for your own sandbox if it exists. Yes, abusers will sooner or later learn this fact and will start exploiting it, but otherwise, you will ban local development and your own website much sooner than that.
|
||||
|
||||
The general conclusion is:
|
||||
* it is highly desirable to have partners formally identified (either through obtaining API keys or by providing contact data such as website domain or application identifier in a store while initializing the API);
|
||||
@ -59,16 +59,17 @@ Usually, you can put forward some requirements for self-identifying of partners,
|
||||
|
||||
1. The most simple and obvious indicator is an IP address. It's very hard to counterfeit them (e.g. the API server always knows the remote address), and the IP address statistics are reasonably demonstrative.
|
||||
|
||||
If the API is provided as a server-to-server one, there will be no access to the end user's IP address. However, it makes sense to require partners to propagate the IP address (for example, in a form of the `X-Forwarder-For` header) — among other things, to help partners fight fraud and unintended usage of the API.
|
||||
If the API is provided as a server-to-server one, there will be no access to the end user's IP address. However, it makes sense to require partners to propagate the IP address (for example, in a form of the `X-Forwarded-For` header) — among other things, to help partners fight fraud and unintended usage of the API.
|
||||
|
||||
Until recently, IP addresses were also a convenient statistics indicator because it was quite expensive to get a large pool of unique addresses. However, with ipv6 advancement this restriction is no longer actual; ipv6 rather put the light on the fact that you can't just count unique addresses — the aggregates are to be tracked:
|
||||
* the cumulative number of requests by networks, e.g. the hierarchical calculations (the number of /8, /16, /24, etc. networks)
|
||||
* the cumulative statistics by autonomous networks (AS);
|
||||
* the API requests through known public proxies and TOR network.
|
||||
|
||||
An abnormal number of requests in one network might be evidence of the API being actively used inside some corporative environment (or in this region NATs are widespread).
|
||||
|
||||
2. Additional means of tracking are users' unique identifiers, most notably cookies. However, most recently this method of gathering data is under attack from several sides: browser makers restrict third-party cookies, users are employing anti-tracker software, and lawmakers started to roll out legal requirements against data collection. In the current situation, it's much easier to drop cookie usage than to be compliant with all the regulations.
|
||||
* the cumulative number of requests by networks, e.g. the hierarchical calculations (the number of /8, /16, /24, etc. networks)
|
||||
* the cumulative statistics by autonomous networks (AS);
|
||||
* the API requests through known public proxies and TOR network.
|
||||
|
||||
An abnormal number of requests in one network might be evidence of the API being actively used inside some corporative environment (or NATs being widespread in the region).
|
||||
|
||||
2. Additional means of tracking are users' unique identifiers, most notably cookies. However, most recently this method of gathering data got attacked from several directions: browser makers restrict third-party cookies, users are employing anti-tracker software, and lawmakers started to roll out legal requirements against data collection. In the current situation, it's much easier to drop cookie usage than to be compliant with all the regulations.
|
||||
|
||||
All this leads to a situation when public APIs (especially those installed on free-to-use sites and applications) are very limited in the means of collecting statistics and analyzing user behavior. And that impacts not only fighting all kinds of fraud but analyzing use cases as well. That's the way.
|
||||
|
||||
|
@ -2,7 +2,7 @@
|
||||
|
||||
Implementing the paradigm of a centralized system of preventing partner endpoints-bound fraud, which we described in the previous chapter, in practice faces non-trivial difficulties.
|
||||
|
||||
The task of filtering out illicit API requests generally comprises three steps:
|
||||
The task of filtering out illicit API requests comprises three steps:
|
||||
* identifying suspicious users;
|
||||
* optionally, asking for an additional authentication factor;
|
||||
* making decisions and applying access restrictions.
|
||||
@ -11,11 +11,11 @@ The task of filtering out illicit API requests generally comprises three steps:
|
||||
|
||||
Generally speaking, there are two approaches we might take, the static one and the dynamic (behavioral) one.
|
||||
|
||||
*Statically* we monitor suspicions activity surges, as described in the previous chapter, marking an unusually high density of requests coming from specific networks or Referer's (actually, *any* piece of information suits if it divides users into more or less independent groups: for example, OS version or system language if you can gather those).
|
||||
*Statically* we monitor suspicions activity surges, as described in the previous chapter, marking an unusually high density of requests coming from specific networks or `Referer`s (actually, *any* piece of information suits if it splits users into more or less independent groups: for example, OS version or system language would suffice if you can gather those).
|
||||
|
||||
*Behavioral* analysis means we're examining the history of requests made by a specific user, searching for non-typical patterns, such as “unhuman” order of traversing endpoints or too small pauses between requests.
|
||||
|
||||
**Importantly**, when we talk about “user,” we will have to make a second analytical contour to work with IP addresses, as malefactors aren't obliged to preserve cookies or other identification tokens, or will keep a pool of such tokens to impede their exposure.
|
||||
**Importantly**, when we talk about “users,” we will have to make duplicate systems to observe them both using tokens (cookies, logins, phone numbers) and IP addresses, as malefactors aren't obliged to preserve the tokens between requests, or might keep a pool of them to impede their exposure.
|
||||
|
||||
##### Requesting an Additional Authentication Factor
|
||||
|
||||
@ -25,17 +25,17 @@ In the case of services for end users, the main method of acquiring the second f
|
||||
|
||||
**NB**. Instead of captcha, there might be other actions introducing additional authentication factors. It might be the phone number confirmation or the second step of the 3D-Secure protocol. The important part is that requesting an additional authentication step must be stipulated in the program interface, as it can't be added later in a backwards-compatible manner.
|
||||
|
||||
Other popular mechanics of identifying robots include offering a bait (“honeypot”) or employing the execution environment checks (starting from rather trivial like executing JavaScript on the webpage and ending with sophisticated techniques of checking application integrity).
|
||||
Other popular mechanics of identifying robots include offering a bait (“honeypot”) or employing the execution environment checks (starting from rather trivial ones like executing JavaScript on the webpage and ending with sophisticated techniques of checking application integrity checksums).
|
||||
|
||||
##### Restricting Access
|
||||
|
||||
The illusion of having a broad choice of technical means of identifying fraud users should not deceive you as you will soon discover the lack of effective methods of restricting those users. Banning them by cookie / Referer / User-Agent makes little to no impact as this data is supplied by clients, and might be easily forged. In the end, you have four mechanisms for suppressing illegal activities:
|
||||
* banning users by IP (networks, autonomous systems);
|
||||
* requiring mandatory user identification (maybe layered: login / login with confirmed phone number / login with confirmed identity / login with confirmed identity and biometrics / etc.);
|
||||
* returning fake responses;
|
||||
The illusion of having a broad choice of technical means of identifying fraud users should not deceive you as you will soon discover the lack of effective methods of restricting those users. Banning them by cookie / `Referer` / `User-Agent` makes little to no impact as this data is supplied by clients, and might be easily forged. In the end, you have four mechanisms for suppressing illegal activities:
|
||||
* banning users by IP (networks, autonomous systems)
|
||||
* requiring mandatory user identification (maybe tiered: login / login with confirmed phone number / login with confirmed identity / login with confirmed identity and biometrics / etc.)
|
||||
* returning fake responses
|
||||
* filing administrative abuse reports.
|
||||
|
||||
The problem with option number one is the collateral damage you will inflict, especially if you have to ban subnets.
|
||||
The problem with the first option is the collateral damage you will inflict, especially if you have to ban subnets.
|
||||
|
||||
The second option, though quite rational, is usually inapplicable to real APIs, as not every partner will agree with the approach, and definitely not every end user. This will also require being compliant with the existing personal data laws.
|
||||
|
||||
@ -43,9 +43,9 @@ The third option is the most effective one in technical terms as it allows to pu
|
||||
|
||||
Thereby, you have only one method that really works: filing complaints to hosting providers, ISPs, or law enforcement authorities. Needless to say, this brings certain reputational risks, and the reaction time is rather not lightning fast.
|
||||
|
||||
In most cases, you're not fighting fraud — you're actually increasing the cost of the attack, simultaneously buying yourself enough time to make administrative moves against the perpetrator. Preventing API misusage completely is impossible as malefactors might ultimately employ the expensive but bulletproof solution — to hire real people to make the requests to the API on real devices through legal applications.
|
||||
In most cases, you're not fighting fraud — you're actually increasing the cost of the attack, simultaneously buying yourself enough time to make administrative moves against the perpetrator. Preventing API misusage completely is impossible as malefactors might ultimately employ the expensive but bulletproof solution — to hire real people to make the requests to the API on real devices through legitimate applications.
|
||||
|
||||
An opinion exists, which the author of this book shares, that engaging in this sword-against-shield confrontation must be carefully thought out, and advanced technical solutions are to be enabled only if you are one hundred percent sure it is worth it (e.g. if they steal real money or data). By introducing elaborate algorithms, you rather conduct an evolutional selection of the smartest and most cunning cybercriminals, counteracting who will be way harder than those who just naively call API endpoints with `curl`. What is even more important, in the final phase — e.g. when filing the complaint to authorities — you will have to prove the alleged ToS violation, and doing so against an advanced fraudster will be problematic. So it's rather better to have all the malefactors monitored (and regularly complained against), and escalate the situation (e.g. enable the technical protection and start legal actions) only if the threat passes a certain threshold. That also implies that you must have all the tools ready, and just keep them below fraudsters' radars.
|
||||
An opinion exists, which the author of this book shares, that engaging in this sword-against-shield confrontation must be carefully thought out, and advanced technical solutions are to be enabled only if you are one hundred percent sure it is worth it (e.g. if they steal real money or data). By introducing elaborate algorithms, you rather conduct an evolutional selection of the smartest and most cunning cybercriminals, counteracting to whom will be way harder than to those who just naïvely call API endpoints with `curl`. What is even more important, in the final phase — e.g. when filing the complaint to authorities — you will have to prove the alleged ToS violation, and doing so against an advanced fraudster will be problematic. So it's rather better to have all the malefactors monitored (and regularly complained against), and escalate the situation (e.g. enable the technical protection and start legal actions) only if the threat passes a certain threshold. That also implies that you must have all the tools ready, and just keep them below fraudsters' radars.
|
||||
|
||||
Out of the author of this book's experience, the mind games with malefactors, when you respond to any improvement of their script with the smallest possible effort that is enough to break it, might continue indefinitely. This strategy, e.g. making fraudsters guess which traits were used to ban them this time (instead of unleashing the whole heavy artillery potential), annoys amateur “hackers” greatly as they lack hard engineering skills and just give up eventually.
|
||||
|
||||
@ -53,11 +53,11 @@ Out of the author of this book's experience, the mind games with malefactors, wh
|
||||
|
||||
Let's now move to the second type of unlawful API usage, namely using in the malefactor's applications keys stolen from conscientious partners. As the requests are generated by real users, captcha won't help, though other techniques will.
|
||||
|
||||
1. Maintaining metrics collection by IP addresses and subnets might be of use in this case as well. If the malefactor's app isn't a public one but rather targeted to some closed audience, this fact will be visible in the dashboards (and if you're lucky enough, you might also find suspicious Referers, public access to which is restricted).
|
||||
1. Maintaining metrics collection by IP addresses and subnets might be of use in this case as well. If the malefactor's app isn't a public one but rather targeted to some closed audience, this fact will be visible in the dashboards (and if you're lucky enough, you might also find suspicious `Referer`s, public access to which is restricted).
|
||||
|
||||
2. Allowing partners to restrict the functionality available under specific API keys:
|
||||
|
||||
* setting the allowed IP address range for server-to-server APIs, allowed Referer's and application ids for client APIs;
|
||||
* setting the allowed IP address range for server-to-server APIs, allowed `Referer`s and application ids for client APIs;
|
||||
|
||||
* white-listing only allowed API functions for a specific key;
|
||||
|
||||
@ -67,7 +67,7 @@ Let's now move to the second type of unlawful API usage, namely using in the mal
|
||||
|
||||
* for example, if on the partner's website, there is a form displaying the best lungo offers, for which the partners call the API endpoint like `/v1/search?recipe=lungo&api_key={apiKey}`, then the API key might be replaced with a signature like `sign = HMAC("recipe=lungo", apiKey)`; the signature might be stolen as well, but it will be useless for malefactors as they will be able to find only lungo with it;
|
||||
|
||||
* instead of API keys, time-based one-time passwords (TOTP) might be used; these tokens are valid for a short period of time (usually, one minute) only, which makes working with stealing keys much more sophisticated.
|
||||
* instead of API keys, time-based one-time passwords (TOTP) might be used; these tokens are valid for a short period of time only (typically, one minute), which makes using stolen keys much more complicated.
|
||||
|
||||
4. Filing complaints to the administration (hosting providers, app store owners) in case the malefactor distributes their application through stores or uses a diligent hosting service that investigates abuse filings. Legal actions are also an option, and even much so compared to countering user fraud, as illegal access to the system using stolen credentials is unambiguously outlawed in most jurisdictions.
|
||||
|
||||
|
@ -1,6 +1,6 @@
|
||||
### [Supporting customers][api-product-customer-support]
|
||||
|
||||
First of all, an important remark: when we talk about supporting API customers, we mean supporting developers and to some extent business partners. End users seldom interact with APIs directly, with an exception of several non-standard cases.
|
||||
From banning users, let's change the topic to supporting them. First of all, an important remark: when we talk about supporting API customers, we mean helping developers and to some extent business partners. End users seldom interact with APIs directly, with an exception of several non-standard cases:
|
||||
|
||||
1. If you can't reach partners that are using the API incorrectly, you might have to display errors that end users can see. This might happen if the API was provided for free and with minimum partner identification requirements while in the growth phase, and then the conditions changed (a popular API version is no longer supported or became paid).
|
||||
|
||||
@ -18,21 +18,21 @@ The former is of course extremely important for any healthy service (including A
|
||||
|
||||
As an API is a program product, developers will be in fact asking how this specific piece of code that they have written works. This fact raises the level of required customer support staff members' expertise quite high as you need a software engineer to read the code and understand the problem. But this is but half of the problem; another half is, as we have mentioned in the previous chapters, that most of these questions will be asked by inexperienced or amateur developers. In a case of a popular API, it means that 9 out of 10 inquiries *will not be about the API*. Less skilled developers lack language knowledge, their experience with the platform is fragmented, and they can't properly formulate their problem (and therefore search for an answer on the Internet before contacting support; though, let us be honest, they usually don't even try).
|
||||
|
||||
There are several options for tackling these issues.
|
||||
There are several options for tackling these issues:
|
||||
|
||||
1. The most user-friendly scenario is hiring people with basic technical skills as the first line of support. These employees must possess enough expertise in understanding how the API works to be able to identify those unrelated questions and respond to them according to some FAQ, point out to the relevant external resource (let's say, the support service of the platform or the community forum of the programming language), or redirect relevant issues to the API developers.
|
||||
1. The most user-friendly scenario is hiring people with basic technical skills as the first line of support. These employees must possess enough expertise in understanding how the API works to be able to identify those unrelated questions and respond to them according to some FAQ, point out to a relevant external resource (let's say, the support service of the OS or the community forum of the programming language) if the problem is not related to the API itself, and redirect relevant issues to the API developers.
|
||||
|
||||
2. The inverse scenario: partners must pay for technical support, and it's the API developers who answer the questions. It doesn't actually make a significant difference in terms of the quality of the issues (it's still inexperienced developers who use the API and ask questions; you will just cut off those who can't afford paid support) but at least you won't have a hiring problem as you might allow yourself the luxury of hiring engineers for the first line of support.
|
||||
2. The inverse scenario: partners must pay for technical support, and it's the API developers who answer the questions. It doesn't actually make a significant difference in terms of the quality of the issues (it's still mostly inexperienced developers who can't solve the problem on their own; you will just cut off those who can't afford paid support) but at least you won't have a hiring problem as you might allow yourself the luxury of having engineers for the first line of support.
|
||||
|
||||
3. Partly (or, sometimes, fully) the developer community might help with solving the amateur problems (see the [“Communicating with Developers”](#api-product-devrel) chapter). Usually, community members are pretty capable of answering those questions, especially if moderators help them.
|
||||
|
||||
Importantly, whatever options you choose, it's still the API developers in the second line of support simply because only they can fully understand the problem and the partners' code. That implies two important consequences.
|
||||
Importantly, whatever options you choose, it's still the API developers in the second line of support simply because only they can fully understand the problem and the partners' code. That implies two important consequences:
|
||||
|
||||
1. You must consider the time needed to investigate inquiries while planning the API development team time. Reading unfamiliar code and remote debugging are very hard and exhausting tasks. The more functionality you expose and the more platforms you support, the more load is put on the team in terms of dealing with support tickets.
|
||||
1. You must take into account working with inquiries while planning the API development team time. Reading unfamiliar code and remote debugging are very hard and exhausting tasks. The more functionality you expose and the more platforms you support, the more load is put on the team in terms of dealing with support tickets.
|
||||
|
||||
2. As a rule, developers are totally not happy with the perspective of sorting the incoming requests and answering them. The first line of support will still let through a lot of dilettante or badly formulated questions, and that will annoy on-duty API developers. There are several approaches to mitigate the problem:
|
||||
2. As a rule, developers are totally not happy about the perspective of coping with incoming requests and answering them. The first line of support will still let through a lot of dilettante or badly formulated questions, and that will annoy on-duty API developers. There are several approaches to mitigate the problem:
|
||||
|
||||
* try to find people with a customer-oriented mindset, who like this activity, and encourage them (including financial stimulus) to deal with support; it might be someone on the team (and not necessarily a developer) or some active community member;
|
||||
* try to find people with a customer-oriented mindset, who like this activity, and encourage them (including financial stimulus) to perform support functions; it might be someone on the team (and not necessarily a developer) or some active community member;
|
||||
|
||||
* the remaining load must be distributed among the developers equally and fairly, up to introducing the duty calendar.
|
||||
|
||||
@ -40,4 +40,4 @@ And of course, analyzing the questions is a useful exercise to populate FAQs and
|
||||
|
||||
#### External Platforms
|
||||
|
||||
Sooner or later, you will find that customers ask their questions not only through the official channels, but also on numerous Internet-based forums, starting from those specifically created for this, like StackOverflow, and ending with social networks and personal blogs. It's up to you whether to spend time searching for such inquiries; we would rather recommend providing support through those sites that have convenient tools for that (like subscribing to specific tags).
|
||||
Sooner or later, you will find that customers ask their questions not only through the official channels, but also on numerous Internet-based forums, starting from those specifically created for this, like StackOverflow, and ending with social networks and personal blogs. It's up to you whether to spend time searching for such inquiries. We would rather recommend providing support through those platforms that have convenient tools for that (like subscribing to specific tags).
|
@ -11,9 +11,9 @@ In fact, newcomers (e.g. those developers who are not familiar with the API) usu
|
||||
|
||||
#### Introductory Notes
|
||||
|
||||
Documentation frequently suffers from being excessively clerical; it's being written using formal terminology (which often requires reading the glossary before the actual docs) and being unreasonably inflated. So instead of a two-word answer to the user's question a couple of paragraphs are conceived — a practice we strongly disapprove of. The perfect documentation must be simple and laconic, and all the terms must be either explained in the text or given a reference to such an explanation. However, “simple” doesn't mean “illiterate”: remember, the documentation is the face of your product, so grammar errors and improper usage of terms are unacceptable.
|
||||
Documentation frequently suffers from being excessively clerical; it's being written using formal terminology (which often requires reading the glossary before the actual docs) and is frequently unreasonably inflated. So instead of a two-word answer to a user's question, a couple of paragraphs is conceived — a practice we strongly disapprove of. The perfect documentation must be simple and laconic, and all the terms must be either explained in the text or given a reference to such an explanation. However, “simple” doesn't mean “illiterate”: remember, the documentation is the face of your product, so grammar errors and improper usage of terms are unacceptable.
|
||||
|
||||
Also, keep in mind that documentation will be used for searching as well, so every page should contain all the keywords required to be properly ranked by search engines. This requirement somehow contradicts the simple-and-laconic principle; that's the way.
|
||||
Also, keep in mind that documentation will be used for searching as well, so every page should contain all the keywords required to be properly ranked by search engines. Unfortunately, this requirement contradicts the simple-and-laconic principle; that's the way.
|
||||
|
||||
#### Documentation Content Types
|
||||
|
||||
@ -23,7 +23,7 @@ Any documentation starts with a formal functional description. This content type
|
||||
|
||||
Today, a reference must be also a machine-readable specification, e.g. comply with some standard, for example, OpenAPI.
|
||||
|
||||
A specification must comprise not only formal descriptions but implicit agreements as well, such as the event generation order or unobvious side-effects of the API methods. Its most important applied value is advisory consulting: developers will refer to it to clarify unobvious situations.
|
||||
The specification must comprise not only formal descriptions but implicit agreements as well, such as the event generation order or unobvious side-effects of the API methods. Its important applied value is advisory consulting: developers will refer to it to clarify unobvious situations.
|
||||
|
||||
**Importantly**, formal specification *is not documentation* per se. The documentation is *the words you write* in the descriptions of each field and method. Without them, the specification might be used just for checking whether your namings are fine enough for developers to guess their meaning.
|
||||
|
||||
@ -54,27 +54,25 @@ A tutorial is a sort of “book” that you write to explain to the reader how t
|
||||
* proper reactions to program errors that could happen;
|
||||
* detailed studies on advanced API functionality (with detailed examples).
|
||||
|
||||
As usual, a tutorial comprises a common section (basic terms and concepts, notation keys) and a set of sections regarding each functional domain exposed via the API.
|
||||
|
||||
Usually, tutorials contain a “Quick Start” (“Hello, world!”) section: the smallest possible code sample that would allow developers to build a small app atop the API. “Quick Starts” aim to cover two needs:
|
||||
Usually, a tutorial comprises a common section (basic terms and concepts, notation keys) and a set of sections regarding each functional domain exposed via the API. Frequently, tutorials contain a “Quick Start” (“Hello, world!”) section: the smallest possible code sample that would allow developers to build a small app atop the API. “Quick Starts” aim to cover two needs:
|
||||
* to provide a default entry-point, the easiest to understand and the most useful text for those who heard about your API for the first time;
|
||||
* to engage developers, to make them touch the service by a mean of a real-world example.
|
||||
|
||||
Also, “Quick starts” are a good indicator of how exactly well did you do your homework of identifying the most important use cases and providing helper methods. If your Quick Start comprises more than ten lines of code, you have definitely done something wrong.
|
||||
|
||||
##### Frequently Asked Questions and a Knowledge Base
|
||||
##### Frequently Asked Questions and Knowledge Bases
|
||||
|
||||
After you publish the API and start supporting users (see the previous chapter) you will also accumulate some knowledge of what questions are asked most frequently. If you can't easily integrate answers into the documentation, it's useful to compile a specific “Frequently Asked Questions” (aka FAQ) article. A FAQ article must meet the following criteria:
|
||||
* address the real questions (you might frequently find FAQs that were reflecting not users' needs, but the API owner's desire to repeat some important information once more; it's useless, or worse — annoying; perfect examples of this anti-pattern realization might be found on any bank or air company website);
|
||||
* both questions and answers must be formulated clearly and succinctly; it's acceptable (and even desirable) to provide links to corresponding reference and tutorial articles, but the answer itself can't be longer than a couple of paragraphs.
|
||||
|
||||
Also, FAQs are a convenient place to explicitly highlight the advantages of the API. In a question-answer form, you might demonstrably show how your API solves complex problems easily and handsomely. (Or at least, solves them generally unlike the competitors' products.)
|
||||
Also, FAQs are a convenient place to explicitly highlight the advantages of the API. In a question-answer form, you might demonstrably show how your API solves complex problems easily and handsomely. (Or at least, *solves them*, unlike the competitors' products.)
|
||||
|
||||
If technical support conversations are public, it makes sense to store all the questions and answers as a separate service to form a knowledge base, e.g. a set of “real-life” questions and answers.
|
||||
|
||||
##### Offline Documentation
|
||||
|
||||
Though we live in the online world, an offline version of the documentation (in a form of a generated doc file) still might be useful — first of all, as a snapshot of the API valid for a specific date.
|
||||
Though we live in the online world, an offline version of the documentation (in a form of a generated doc file) still might be useful — first of all, as a snapshot of the API specification valid for a specific date.
|
||||
|
||||
#### Content Duplication Problems
|
||||
|
||||
|
@ -4,7 +4,7 @@ If the operations executed via the API imply consequences for end users or partn
|
||||
|
||||
However, in many cases having a test version is not enough — like in our coffee-machine API example. If an order is created but not served, partners are not able to test the functionality of delivering the order or requesting a refund. To run the full cycle of testing, developers need the capability of pushing the order through stages, as this would happen in reality.
|
||||
|
||||
A direct solution to this problem is providing a full set of testing APIs and administrative interfaces. It means that developers will need to run a second application in parallel — the one you're giving to coffee shops so they might get and serve orders (and if there is a delivery functionality, the third app as well: the courier's one) — and make all these actions that coffee shop staff normally does. Obviously, that's not an ideal solution, because of several reasons:
|
||||
A direct solution to this problem is providing test versions for a full set of APIs and administrative interfaces. It means that developers will be able to run a second application in parallel — the one you're giving to coffee shops so they might get and serve orders (and if there is a delivery functionality, the third app as well: the courier's one) — and make all these actions that coffee shop staff normally does. Obviously, that's not an ideal solution, because of several reasons:
|
||||
* developers of end user applications will need to additionally learn how coffee shop and courier apps work, which has nothing to do with the task they're solving;
|
||||
* you will need to invent and implement some matching algorithm: an order made through a test application must be assigned to a specific virtual courier; this actually means creating an isolated virtual “sandbox” (meaning — a full set of services) for each specific partner;
|
||||
* executing a full “happy path” of an order will take minutes, maybe tens of minutes, and will require making a multitude of actions in several different interfaces.
|
||||
@ -21,7 +21,7 @@ The disadvantage of this approach is that client developers still need to know h
|
||||
|
||||
##### The Simulator of Pre-Defined Scenarios
|
||||
|
||||
The alternative to providing the testing environment API is simulating the working scenarios. In this case, the testing environment takes control over “underwater” parts of the system and “plays” all external agents' actions. In our coffee example, that means that, after the order is submitted, the system will simulate all the preparation steps and then the delivery of the beverage to the customer.
|
||||
The alternative to providing the testing environment API is simulating the working scenarios. In this case, the testing environment takes control over “underwater” parts of the system and “plays out” all external agents' actions. In our coffee example, that means that, after the order is submitted, the system will simulate all the preparation steps and then the delivery of the beverage to the customer.
|
||||
|
||||
The advantage of this approach is that it demonstrates vividly how the system works according to the API vendor design plans, e.g. in which sequence the events are generated, and which stages the order passes through. It also reduces the chance of making mistakes in testing scripts, as the API vendor guarantees the actions will be executed in the correct order with the right parameters.
|
||||
|
||||
@ -31,4 +31,4 @@ The main disadvantage is the necessity to create a separate scenario for each un
|
||||
|
||||
Your final goal in implementing testing APIs, regardless of which option you choose, is allowing partners to automate the QA process for their products. The testing environment should be developed with this purpose in mind; for example, if an end user might be brought to a 3-D Secure page to pay for the order, the testing environment API must provide some way of simulating the successful (or not) passing of this step. Also, in both variants, it's possible (and desirable) to allow running the scenarios in a fast-forward manner that will allow making auto-testing much faster than manual testing.
|
||||
|
||||
Of course, not every partner will be able to employ this possibility (which also means that a “manual” way of testing usage scenarios must also be supported alongside the programmatical one) simply because not every business might afford to hire a QA automation engineer. Nevertheless, the ability to write such auto-tests is your API's huge competitive advantage from a technically advanced partner's point of view.
|
||||
Of course, not every partner will be able to employ this possibility (which also means that a “manual” way of testing usage scenarios must always be supported alongside the programmatical one) simply because not every business might afford to hire a QA automation engineer. Nevertheless, the ability to write such auto-tests is your API's huge competitive advantage from a technically advanced partner's point of view.
|
@ -6,17 +6,15 @@ Finally, the last aspect we would like to shed the light on is managing partners
|
||||
|
||||
Ideally, the API once published should live eternally; but as we all are reasonable people, we do understand it's impossible in the real life. Even if we continue supporting older versions, they will still become outdated eventually, and partners will need to rewrite the code to use newer functionality.
|
||||
|
||||
The author of this book formulates the rule of issuing new major API versions like this: the period of time after which partners will need to rewrite the code should coincide with the application lifespan in the subject area. Any program will once become obsolete and will be rewritten; and if during this re-developing partners need to also switch to a newer API version, it will be met with some degree of understanding. Of course, in different subject areas, this timespan differs, depending on the evolution rate of the underlying platform.
|
||||
The author of this book formulates the rule of issuing new major API versions like this: the period of time after which partners will need to rewrite the code should coincide with the application lifespan in the subject area (see [“The Backwards Compatibility Problem Statement”](#back-compat-statement) chapter). Apart from updating *major* versions, sooner or later you will face issues with accessing some outdated *minor* versions as well. As we mentioned in the [“On the Waterline of the Iceberg”](#back-compat-iceberg-waterline) chapter, even fixing bugs might eventually lead to breaking some integrations, and that naturally leads us to the necessity of keeping older *minor* versions of the API until the partner resolves the problem.
|
||||
|
||||
Apart from updating *major* versions, sooner or later you will face issues with accessing some outdated *minor* versions as well. As we mentioned in the [“On the Waterline of the Iceberg”](#back-compat-iceberg-waterline) chapter, even fixing bugs might eventually lead to breaking some integrations, and that naturally leads us to the necessity of keeping older *minor* versions of the API until the partner resolves the problem.
|
||||
|
||||
In this aspect, integrating with large companies that have a dedicated software engineering department differs dramatically from providing a solution to individual amateur programmers: from one side, the former are much more likely to find undocumented features and unfixed bugs in your code; on the other side, because of the internal bureaucracy, fixing the related issues might easily take months, save not years. The common recommendation there is to maintain old minor API versions for a period of time long enough for the most dilatory partner to switch no the newest version.
|
||||
In this aspect, integrating with large companies that have a dedicated software engineering department differs dramatically from providing a solution to individual amateur programmers: on one hand, the former are much more likely to find undocumented features and unfixed bugs in your code; on the other hand, because of the internal bureaucracy, fixing the related issues might easily take months, save not years. The common recommendation there is to maintain old minor API versions for a period of time long enough for the most dilatory partner to switch no the newest version.
|
||||
|
||||
#### Supporting Platforms
|
||||
|
||||
Another aspect crucial to interacting with large integrators is supporting a zoo of platforms (browsers, programming languages, protocols, operating systems) and their versions. As usual, big companies have their own policies on which platforms they support, and these policies might sometimes contradict common sense. (Let's say, it's rather a time to abandon TLS 1.2, but many integrators continue working through this protocol, or even the earlier ones.)
|
||||
|
||||
Formally speaking, ceasing support of a platform *is* a backwards-incompatible change, and might lead to breaking some integration for some end users. So it's highly important to have clearly formulated policies on which platforms are supported based on which criteria. In the case of mass public APIs, that's usually simple (like, API vendor promises to support platforms that have more than N% penetration, or, even easier, just last M versions of a platform); in the case of commercial APIs, it's always a bargain based on the estimations, how much will non-supporting a specific platform would cost to a company. And of course, the outcome of the bargain must be stated in the contracts — what exactly you're promising to support during which period of time.
|
||||
Formally speaking, ceasing support of a platform *is* a backwards-incompatible change, and might lead to breaking some integration for some end users. So it's highly important to have clearly formulated policies regarding which platforms are supported based on which criteria. In the case of mass public APIs, that's usually simple (like, API vendor promises to support platforms that have more than N% penetration, or, even easier, just last M versions of a platform); in the case of commercial APIs, it's always a bargain based on the estimations, how much will non-supporting a specific platform would cost to a company. And of course, the outcome of the bargain must be stated in the contracts — what exactly you're promising to support during which period of time.
|
||||
|
||||
#### Moving Forward
|
||||
|
||||
@ -24,4 +22,4 @@ Finally, apart from those specific issues, your customers must be caring about m
|
||||
|
||||
You might work with these customer expectations by publishing roadmaps. It's quite common that many companies avoid publicly announcing their concrete plans (for a reason, of course). Nevertheless, in the case of APIs, we strongly recommend providing the roadmaps, even if they are tentative and lack precise dates — *especially* if we talk about deprecating some functionality. Announcing these promises (given the company keeps them, of course) is a very important competitive advantage to every kind of consumer.
|
||||
|
||||
With this, we would like to conclude this book. We hope that the principles and the concepts we have outlined will help you in creating APIs that fit all the developer', business', and end user's needs, and in expanding them (while maintaining the backwards compatibility) for the next two thousand years (or maybe more).
|
||||
With this, we would like to conclude this book. We hope that the principles and the concepts we have outlined will help you in creating APIs that fit all the developers, businesses, and end users' needs, and in expanding them (while maintaining the backwards compatibility) for the next two thousand years or so.
|
@ -41,13 +41,13 @@
|
||||
|
||||
Так или иначе, встаёт вопрос независимой валидации: каким образом можно проконтролировать, действительно ли API используется потребителем в соответствии с пользовательским соглашением.
|
||||
|
||||
Мобильные приложения удобно отслеживаются по идентификатору приложения в соответствующем сторе (Google Play, App Store и другие), поэтому разумно требовать от партнёров идентифицировать приложение при подключении API. Вебсайты с некоторой точностью можно идентифицировать по заголовкам Referer или Origin (и для надёжности можно так же потребовать от партнёра указывать домен сайта при инициализации API).
|
||||
Мобильные приложения удобно отслеживаются по идентификатору приложения в соответствующем сторе (Google Play, App Store и другие), поэтому разумно требовать от партнёров идентифицировать приложение при подключении API. Вебсайты с некоторой точностью можно идентифицировать по заголовкам `Referer` или `Origin` (и для надёжности можно так же потребовать от партнёра указывать домен сайта при инициализации API).
|
||||
|
||||
Эти данные сами по себе не являются надёжными; важно то, что они позволяют проводить кросс-проверки:
|
||||
* если ключ был выпущен для одного домена, но запросы приходят с Referer-ом другого домена — это повод разобраться в ситуации и, возможно, забанить возможность обращаться к API с этим Referer-ом или этим ключом;
|
||||
* если ключ был выпущен для одного домена, но запросы приходят с `Referer`-ом другого домена — это повод разобраться в ситуации и, возможно, забанить возможность обращаться к API с этим `Referer`-ом или этим ключом;
|
||||
* если одно приложение инициализирует API с указанием ключа другого приложения — это повод обратиться к администрации стора с требованием удалить одно из приложений.
|
||||
|
||||
**NB**: не забудьте разрешить безлимитное использование с Referer-ом `localhost` и `127.0.0.1` / `[::1]`, а также из вашей собственной песочницы, если она есть. Да, в какой-то момент злоумышленники поймут, что на такие Referer-ы не действуют ограничения, но это точно произойдёт гораздо позже, чем вы по неосторожности забаните локальную разработку или собственный сайт документации.
|
||||
**NB**: не забудьте разрешить безлимитное использование с `Referer`-ом `localhost` и `127.0.0.1` / `[::1]`, а также из вашей собственной песочницы, если она есть. Да, в какой-то момент злоумышленники поймут, что на такие `Referer`-ы не действуют ограничения, но это точно произойдёт гораздо позже, чем вы по неосторожности забаните локальную разработку или собственный сайт документации.
|
||||
|
||||
Общий вывод из вышеизложенного таков:
|
||||
* очень желательно иметь формальную идентификацию пользователей (API-ключи как самая распространённая практика, либо указание контактных данных, таких как домен вебсайта или идентификатор приложения в сторе, при инициализации API);
|
||||
@ -57,9 +57,9 @@
|
||||
|
||||
Если к партнёрам вы можете предъявлять какие-то требования по самоидентификации, то от конечных пользователей требовать раскрытия информации о себе в большинстве случаев не представляется возможным. Все методы контроля, описанные ниже, являются неточными и зачастую эвристическими. (Даже если функциональность партнёрских приложений предоставляется только после регистрации пользователя и вы имеете к этой регистрации доступ, вы всё ещё гадаете, т.к. аккаунт это не то же самое, что и отдельный пользователь: несколько различных людей могут пользоваться одним профилем или, наоборот, у одного человека может быть множество профилей.) Кроме того, следует иметь в виду, что сбор подобного рода информации может регулироваться законодательно (хотя большей частью речь пойдёт об анонимизированных данных, но и они могут быть регламентированы).
|
||||
|
||||
1. Самый простой и очевидный показатель — это ip-адреса; их невозможно подделать (в том смысле, что сервер API всегда знает адрес вызывающего клиента), и поэтому статистика по уникальным ip довольно показательна.
|
||||
1. Самый простой и очевидный показатель — это IP-адреса; их невозможно подделать (в том смысле, что сервер API всегда знает адрес вызывающего клиента), и поэтому статистика по уникальным IP довольно показательна.
|
||||
|
||||
Если API предоставляется как server-to-server сервис, доступа к IP-адресу конечного пользователя может и не быть, однако весьма разумно в такой ситуации требовать от партнёра пробрасывать IP-адрес клиента (например, в виде заголовка X-Forwarded-For) — в том числе для того, чтобы помочь партнёрам бороться с фродом и неправомерным использованием API.
|
||||
Если API предоставляется как server-to-server сервис, доступа к IP-адресу конечного пользователя может и не быть, однако весьма разумно в такой ситуации требовать от партнёра пробрасывать IP-адрес клиента (например, в виде заголовка `X-Forwarded-For`) — в том числе для того, чтобы помочь партнёрам бороться с фродом и неправомерным использованием API.
|
||||
|
||||
До недавнего времени ip-адрес как единица подсчёта статистики был ещё и удобен тем, что обзавестись большим пулом уникальных адресов было достаточно дорого. Однако с распространением ipv6 это ограничение перестало быть актуальным; скорее, ipv6 ярко подсветил тот факт, что не стоит ограничиваться только подсчётом уникальных ip. Необходимо следить за несколькими агрегатами:
|
||||
|
||||
|
@ -11,7 +11,7 @@
|
||||
|
||||
По большому счёту, здесь есть всего два подхода, которые мы можем применить — статический и динамический (поведенческий).
|
||||
|
||||
*Статически* мы отслеживаем подозрительную концентрацию активности (как описано в предыдущей главе), отмечая нехарактерно высокое количество запросов из определённых подсетей или Referer-ов (на самом деле, нам подойдёт *любая* информация, которая как-то делит пользователей на более-менее независимые группы — такая как, например, версия ОС или язык системы, если такие данные нам доступны).
|
||||
*Статически* мы отслеживаем подозрительную концентрацию активности (как описано в предыдущей главе), отмечая нехарактерно высокое количество запросов из определённых подсетей или `Referer`-ов (на самом деле, нам подойдёт *любая* информация, которая как-то делит пользователей на более-менее независимые группы — такая как, например, версия ОС или язык системы, если такие данные нам доступны).
|
||||
|
||||
При *поведенческом* анализе мы анализируем историю запросов одного конкретного пользователя и отмечаем нетипичное поведение — «нечеловеческий» порядок обхода эндпойнтов, слишком быстрый их перебор, etc.
|
||||
|
||||
@ -29,7 +29,7 @@
|
||||
|
||||
##### Ограничение доступа
|
||||
|
||||
Видимость богатства способов технической идентификации пользователей, увы, разбивается о суровую реальность наличия у вас очень скромных средств ограничения доступа. Бан по cookie / Referer-у / User-Agent-у практически не работает по той причине, что эти данные передаёт клиент, и он же легко может их подменить. По большому счёту, способов ограничения доступа у вас четыре:
|
||||
Видимость богатства способов технической идентификации пользователей, увы, разбивается о суровую реальность наличия у вас очень скромных средств ограничения доступа. Бан по cookie / `Referer`-у / `User-Agent`-у практически не работает по той причине, что эти данные передаёт клиент, и он же легко может их подменить. По большому счёту, способов ограничения доступа у вас четыре:
|
||||
* бан пользователя по ip (подсети, автономной системе);
|
||||
* требование обязательной идентификации пользователя (возможно, прогрессивной: логин в системе / логин с подтверждённым номером телефона / логин с подтверждением личности / логин с подтверждением личности и биометрией);
|
||||
* отдача ложного ответа;
|
||||
@ -53,7 +53,7 @@
|
||||
|
||||
Рассмотрим теперь второй вариант несанкционированного использования, когда злоумышленник крадёт API-ключ добросовестного партнёра и вставляет его в своё приложение. Запросы при этом генерируются настоящими пользователями, а значит капча никак не поможет — но помогут другие методы.
|
||||
|
||||
1. Ведение статистики по ip-адресам и сетям может помочь и здесь. Если приложение злоумышленника всё-таки не обычное приложение для честных потребителей, а какой-то закрытый сервис для ограниченного круга пользователей, этот факт будет виден на приборах (а если повезёт — то вы увидите ещё и подозрительные Referer-ы, закрытые для внешнего доступа).
|
||||
1. Ведение статистики по ip-адресам и сетям может помочь и здесь. Если приложение злоумышленника всё-таки не обычное приложение для честных потребителей, а какой-то закрытый сервис для ограниченного круга пользователей, этот факт будет виден на приборах (а если повезёт — то вы увидите ещё и подозрительные `Referer`-ы, закрытые для внешнего доступа).
|
||||
|
||||
2. Предоставление возможности партнёрам ограничивать функциональность, которая доступна по ключу:
|
||||
|
||||
|
@ -6,9 +6,7 @@
|
||||
|
||||
Конечно, в идеальном случае однажды выпущенный API должен жить вечно; но, как разумные люди, мы понимаем, что в реальной жизни это невозможно. Даже если мы продолжаем поддерживать старые версии, они все равно морально устаревают: партнёры должны потратить ресурсы на переписывание кода под новые версии API, если хотят получить доступ к новой функциональности.
|
||||
|
||||
Автор этой книги формулирует для себя золотое правило выпуска новых версий API так: период времени, после которого партнеру потребуется переписать свой код, должен совпадать с жизненным циклом приложений в вашей предметной области. Любая программа рано или поздно устареет и будет переписана; если в рамках этого переписывания будет осуществлён и переход на свежую версию API, партнёр, скорее всего, отнесётся к этому с пониманием. Разумеется, в разных областях этот интервал различен, и зависит от скорости эволюции самой нижележащей платформы.
|
||||
|
||||
Помимо переключения *мажорных* версий API, рано или поздно встанет вопрос и о доступе к *минорным* версиям. Как мы упоминали в главе [«О ватерлинии айсберга»](#back-compat-iceberg-waterline), даже исправление ошибок в коде может привести к неработоспособности какой-то интеграции. Соответственно, может потребоваться и сохранение возможности зафиксировать *минорную* версию API до момента обновления затронутого кода партнёром.
|
||||
Автор этой книги формулирует для себя золотое правило выпуска новых версий API так: период времени, после которого партнеру потребуется переписать свой код, должен совпадать с жизненным циклом приложений в вашей предметной области (см. главу [«Постановка проблемы обратной совместимости»](#back-compat-statement)). Помимо переключения *мажорных* версий API, рано или поздно встанет вопрос и о доступе к *минорным* версиям. Как мы упоминали в главе [«О ватерлинии айсберга»](#back-compat-iceberg-waterline), даже исправление ошибок в коде может привести к неработоспособности какой-то интеграции. Соответственно, может потребоваться и сохранение возможности зафиксировать *минорную* версию API до момента обновления затронутого кода партнёром.
|
||||
|
||||
В этом аспекте интеграция с крупными компаниями, имеющими собственный отдел разработки, существенно отличается от взаимодействия с одиночными разработчиками-любителями: первые, с одной стороны, с гораздо большей вероятностью найдут в вашем API недокументированные возможности и неисправленные ошибки; с другой стороны, в силу большей бюрократичности внутренних процессов, исправление проблем может затянуться на месяцы, а то и годы. Общая рекомендация здесь — поддерживать возможность подключения старых минорных версий API достаточно долго для того, чтобы самый забюрократизированный партнёр успел переключиться на новую версию.
|
||||
|
||||
|
Loading…
x
Reference in New Issue
Block a user