books/The-API-Book
1
0
Fork 0
mirror of https://github.com/twirl/The-API-Book.git synced 2025-07-12 22:50:21 +02:00
Code Issues Releases Activity

proofreading

Browse Source
This commit is contained in:
Sergey Konstantinov
2023-09-04 00:35:35 +03:00
parent 8ce1001207
commit 922f927ef5
1 changed files with 20 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
src/en/clean-copy/07-Section VI. The API Product/07.md
View File

@ -1,49 +1,49 @@
### [API Key Performance Indicators][api-product-kpi]
As we described in the previous chapters, there are many API monetization models, both direct and indirect. Importantly, most of them are fully or conditionally free for partners, and the direct-to-indirect benefits ratio tends to change during the API lifecycle. That naturally leads us to the question of how exactly shall we measure the API success and what goals are to be set for the product team.
As we described in the previous chapters, there are many API monetization models, both direct and indirect. Importantly, most of them are fully or conditionally free for partners, and the direct-to-indirect benefits ratio tends to change during the API lifecycle. That naturally leads us to the question of how exactly shall we measure the API's success and what goals are to be set for the product team.
Of course, the most explicit metric is money: if your API is monetized directly or attracts visitors to a monetized service, the rest of the chapter will be of little interest to you, maybe just as a case study. If, however, the contribution of the API to the company's income cannot be simply measured, you have to stick to other, synthetic, indicators.
The obvious key performance indicator (KPI) \#1 is the number of end users and the number of integrations (i.e., partners using the API). Normally, they are in some sense a business health barometer: if there is a normal competitive situation among the API suppliers, and all of them are more or less in the same position, then the figure of how many developers (and consequently, how many end users) are using the API is the main metric of success of the API product.
The obvious key performance indicator (KPI) \#1 is the number of end users and the number of integrations (i.e., partners using the API). Normally, they are in some sense a business health barometer: if there is a normal competitive situation among the API suppliers, and all of them are more or less in the same position, then the figure of how many developers (and consequently, how many end users) are using the API is the main metric of success for the API product.
However, sheer numbers might be deceiving, especially if we talk about free-to-use integrations. There are several factors that make them less reliable:
* The high-level API services that are meant for point-and-click integration (see the previous chapter) are significantly distorting the statistics, especially if the competitors don't provide such services; typically, for one full-scale integration there will be tens, maybe hundreds, of those lightweight embedded widgets.
* Thereby, it's crucial to have partners counted for each kind of the integration independently.
* Thereby, it's crucial to have partners counted for each kind of integration independently.
* Partners tend to use the API in suboptimal ways:
* Embed it at every website page / application screen instead of only those where end users can really interact with the API
* Embed it on every website page / application screen instead of only those where end users can really interact with the API
* Put widgets somewhere deep in the page / screen footer, or hide it behind spoilers
* Initialize a broad range of API modules, but use only a limited subset of them.
* The greater the API audience is, the less the number of unique visitors means as at some moment the penetration will be close to 100%; for example, a regular Internet user interacts with Google or Facebook counters, well, every minute, so the daily audience of those API fundamentally cannot be increased further.
* Initialize a broad range of API modules but use only a limited subset of them.
* The greater the API audience is, the less the number of unique visitors means as at some moment the penetration will be close to 100%; for example, a regular Internet user interacts with Google or Facebook counters, well, every minute, so the daily audience of those APIs fundamentally cannot be increased further.
All the abovementioned problems naturally lead us to a very simple conclusion: not only the raw numbers of users and partners are to be gauged, but their engagement as well, i.e., the target actions (such as searching, observing some data, interacting with widgets) shall be determined and counted. Ideally, these target actions must correlate with the API monetization model:
All the abovementioned problems naturally lead us to a very simple conclusion: not only should the raw numbers of users and partners be gauged, but their engagement as well, i.e., the target actions (such as searching, observing specific data, interacting with widgets) should be determined and counted. Ideally, these target actions must correlate with the API monetization model:
* If the API is monetized through displaying ads, then the user's activity towards those ads (e.g., clicks, interactions) is to be measured.
* If the API attracts customers to the core service, then count the transitions.
* If the API is needed for collecting feedback and gathering UGC, then calculate the number of reviews left and entities edited.
Additionally, the functional KPIs are often employed: how frequently some API features are used. (Also, it helps with prioritizing further API improvements.) In fact, that's still measuring target actions, but those that are made by developers, not end users. It's rather complicated to gather the usage data for software libraries and frameworks, though still doable (however, you must be extremely cautious with that, as any audience rather nervously reacts to finding that some statistic is gathered automatically).
Additionally, functional KPIs are often employed: how frequently some API features are used. (Also, it helps with prioritizing further API improvements.) In fact, that's still measuring target actions, but those made by developers, not end users. It's rather complicated to gather usage data for software libraries and frameworks, though still doable (however, you must be extremely cautious with that, as any audience rather nervously reacts to finding that some statistics are gathered automatically).
The most complicated case is that of API being a tool for (tech)PR and (tech)marketing. In this case, there is a cumulative effect: increasing the API audience doesn't momentarily bring any profit to the company. *First*, you got a loyal developer community, *then* this reputation helps you to hire people. *First*, your company's logo flashes on third-party webpages and applications, *then* the top-of-mind brand knowledge increases. There is no direct method of evaluating how some action (let's say, a new release or an event for developers) affects the target metrics. In this case, you have to operate indirect metrics, such as the audience of the documentation site, the number of mentions in the relevant communication channels, the popularity of your blogs and seminars, etc.
The most complicated case is when the API is a tool for (tech)PR and (tech)marketing. In this case, there is a cumulative effect: increasing the API audience doesn't immediately bring any profit to the company. *First*, you build a loyal developer community, *then* this reputation helps you hire people. *First*, your company's logo flashes on third-party webpages and applications, *then* top-of-mind brand awareness increases. There is no direct method of evaluating how some action (let's say, a new release or an event for developers) affects the target metrics. In this case, you have to operate with indirect metrics, such as the audience of the documentation site, the number of mentions in relevant communication channels, the popularity of your blogs and seminars, etc.
Let us summarize the paragraph:
* Counting direct metrics such as the total number of users and partners is a must and is totally necessary for moving further, but that's not a proper KPI.
* The proper KPI should be formulated based on the number of target actions that are made through the platform.
* The definition of target action depends on the monetization model and might be quite straightforward (like the number of paying partners, or the number of paid ad clicks) or, to the contrary, pretty implicit (like the growth of the company's developer blog audience).
To summarize the paragraph:
* Counting direct metrics such as the total number of users and partners is a must and is absolutely necessary for moving forward, but that's not a proper KPI.
* The proper KPI should be formulated based on the number of target actions made through the platform.
* The definition of target action depends on the monetization model and might be quite straightforward (like the number of paying partners or the number of paid ad clicks) or, conversely, pretty implicit (like the growth of the company's developer blog audience).
#### SLA
This chapter would be incomplete if we didn't mention the “hygienic” KPI — the service level and the service availability. We won't be describing the concept in detail, as the API SLA isn't any different from any other digital services SLAs. Let us just state that this metric must be tracked, especially if we talk about pay-to-use APIs. However, in many cases, API vendors prefer to offer rather loose SLAs, treating the provided functionality as a data access or content licensing service.
This chapter would be incomplete if we didn't mention the “hygienic” KPI — service level and availability. We won't describe the concept in detail, as the API SLA isn't any different from SLAs for other digital services. Let us just state that this metric must be tracked, especially if we talk about pay-to-use APIs. However, in many cases, API vendors prefer to offer rather loose SLAs, treating the provided functionality as data access or content licensing services.
Still, let us re-iterate once more: any problems with your API are automatically multiplied by the number of partners you have, especially if the API is vital for them, i.e., the API outage makes the main functionality of their services unavailable. (And actually, because of the above-mentioned reasons, the average quality of integrations implies that partners' services will suffer even if the availability of the API is not formally speaking critical for them, but because developers use it excessively and do not bother with proper error handling.)
Still, let us reiterate once more: any problems with your API are automatically multiplied by the number of partners you have, especially if the API is vital for them, i.e., the API outage makes the main functionality of their services unavailable. (And actually, because of the above-mentioned reasons, the average quality of integrations implies that partners' services will suffer even if the availability of the API is not formally speaking critical for them, but because developers use it excessively and do not bother with proper error handling.)
It is important to mention that predicting the workload for the API service is rather complicated. Sub-optimal API usage, e.g., initializing the API in those application and website parts where it's not actually needed, might lead to a colossal increase in the number of requests after changing a single line of partner's code. The safety margin for an API service must be much higher than for a regular service for end users — it must survive the situation of the largest partner suddenly starting querying the API on every page and every application screen. (If the partner is already doing that, then the API must survive doubling the load if the partner by accident starts initializing the API twice on each page / screen.)
It is important to mention that predicting the workload for the API service is rather complicated. Sub-optimal API usage, e.g., initializing the API in those parts of applications and websites where it's not actually needed, might lead to a colossal increase in the number of requests after changing a single line of a partner's code. The safety margin for an API service must be much higher than for a regular service for end users — it must survive the situation of the largest partner suddenly starting to query the API on every page and every application screen. (If the partner is already doing that, then the API must survive doubling the load if the partner accidentally starts initializing the API twice on each page / screen.)
Another extremely important hygienic minimum is the informational security of the API service. In the worst-case scenario, namely, if an API service vulnerability allows for exploiting partner applications, one security loophole will in fact be exposed *in every partner application*. Needless to say that the cost of such a mistake might be overwhelmingly colossal, even if the API itself is rather trivial and has no access to sensitive data (especially if we talk about webpages where no “sandbox” for third-party scripts exists, and any piece of code might let's say track the data entered in forms). API services must provide the maximum protection level (for example, choose cryptographical protocols with a certain overhead) and promptly react to any reports regarding possible vulnerabilities.
Another extremely important hygienic minimum is the informational security of the API service. In the worst-case scenario, namely, if an API service vulnerability allows for exploiting partner applications, one security loophole will in fact be exposed *in every partner application*. Needless to say, the cost of such a mistake might be overwhelmingly colossal, even if the API itself is rather trivial and has no access to sensitive data (especially if we talk about webpages where no “sandbox” for third-party scripts exists, and any piece of code might, for example, track the data entered in forms). API services must provide the maximum level of protection (e.g., choose cryptographic protocols with a certain overhead) and promptly react to any reports regarding possible vulnerabilities.
#### Comparing to Competitors
While measuring KPIs of any service, it's important not only to evaluate your own numbers but also to match them against the state of the market:
While measuring KPIs of any service, it's important not only to evaluate your own numbers but also to compare them against the state of the market:
* What is your market share, and how is it evolving over time?
* Is your service growing faster than the market itself or is the rate the same, or is it even less?
* Is your service growing faster than the market itself, or is the growth rate the same, or is it even less?
* What proportion of the growth is caused by the growth of the market, and what is related to your efforts?
Getting answers to those questions might be quite non-trivial in the case of API services. Indeed, how could you learn how many integrations has your competitor had during the same period of time, and what number of target actions had happened on their platform? Sometimes, the providers of popular analytical tools might help you with this, but usually, you have to monitor the potential partners' apps and websites and gather the statistics regarding APIs they're using. The same applies to market research: unless your niche is significant enough for some analytical company to conduct a study, you will have to either commission such work or make your own estimations — conversely, through interviewing potential customers.
Getting answers to those questions might be quite non-trivial in the case of API services. Indeed, how could you learn how many integrations your competitor had during the same period, and what number of target actions had happened on their platform? Sometimes, the providers of popular analytical tools might help you with this, but usually, you have to monitor potential partners' apps and websites and gather statistics regarding the APIs they're using. The same applies to market research: unless your niche is significant enough for some analytical company to conduct a study, you will have to either commission such work or make your own estimations — conversely, through interviewing potential customers.