self-hosted/joplin
1
0
Fork 0
mirror of https://github.com/laurent22/joplin.git synced 2025-11-06 09:19:22 +02:00
Code Issues Releases Activity

All: Add support for application plugins (#3257)

Browse Source
This commit is contained in:
Laurent
2020-10-09 18:35:46 +01:00
committed by GitHub
parent 833fb1264f
commit fe41d37f8f
804 changed files with 95622 additions and 5307 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
readme/api/get_started/plugins.md Normal file
View File

@@ -0,0 +1,47 @@
# Getting started with plugin development
In this article you will learn the basic steps to build and test a plugin in Joplin.
## Setting up your environment
First you need to setup your environment:
- Make sure you have [Node.js](https://nodejs.org/) and [git](https://git-scm.com) installed.
- Install Joplin and run it in development mode.
You will also need to have Joplin installed and running in development mode, which we'll describe later.
But first install Yeoman and the Joplin Plugin Generator:
npm install -g yo generator-joplin
Then to create the plugin, run:
yo joplin
This will create the basic scafolding of the plugin. At the root of it, there is a number of configuration files which you normally won't need to change. Then the `src/` directory will contain your code. By default, the project uses TypeScript, but you are free to use plain JavaScript too - eventually the project is compiled to plain JS in any case.
The `src/` directory also contains a [manifest.json](https://github.com/laurent22/joplin/blob/dev/readme/api/references/plugin_manifest/) file, which you can edit to set various information about the plugin, such as its name, homepage URL, etc.
## Building the plugin
The file `src/index.ts` already contain some basic code meant for testing the plugin. In particular it contains a call to [joplin.plugins.register](https://joplinapp.org/plugins/api/classes/joplinplugins.html), which all plugins should call to register the plugin. And an `onStart()` event handler, which will be executed by Joplin when the plugin starts.
To try this basic plugin, compile the app by running the following from the root of the project:
npm run dist
Doing so should compile all the files into the `dist/` directory. This is from here that Joplin will load the plugin.
## Testing the plugin
In order to test the plugin, you might want to run Joplin in [Development Mode](https://github.com/laurent22/joplin/blob/dev/readme/api/references/development_mode/). Doing so means that Joplin will run using a different profile, so you can experiment with the plugin without risking to accidentally change or delete your data.
Finally, in order to test the plugin, open the Setting screen, then navigate the the **Plugins** section, and add the plugin path in the **Development plugins** text field. For example, if your plugin project path is `/home/user/src/joplin-plugin`, add this in the text field.
Restart the app, and Joplin should load the plugin and execute its `onStart` handler. If all went well you should see the test message in the plugin console: "Test plugin started!".
# Next steps
- You might want to check the [plugin tutorial](https://github.com/laurent22/joplin/blob/dev/readme/api/tutorials/toc_plugin/) to get a good overview of how to create a complete plugin and how to use the plugin API.
- For more information about the plugin API, check the [Plugin API reference](https://joplinapp.org/plugins/api/classes/joplin.html).

18
readme/api/overview.md Normal file
View File

@@ -0,0 +1,18 @@
# Extending Joplin
Joplin provides a number of extension points to allow third-party applications to access its data, or to develop plugins.
The two main extension points are:
- The [data API](https://github.com/laurent22/joplin/blob/dev/readme/api/references/rest_api.md), which is a server that provides access to Joplin data to external applications. It is possible, using standard HTTP calls, to create, modify or delete notes, notebooks, tags, etc. as well as attach files to notes and retrieve these files. This is for example how the web clipper communicates with Joplin.
- The [plugin API](https://joplinapp.org/plugins/api/classes/joplin.html), which allows directly modifying Joplin by adding new features to the application. Using this API, you can:
- Access notes, folders, etc. via the data API
- Add a view to display custom data using HTML/CSS/JS
- Create a dialog to display information and get input from the user
- Create a new command and associate a toolbar button or menu item with it
- Get access to the note currently being edited and modify it
- Listen to various events and run code when they happen
- Hook into the application to set additional options and customise Joplin's behaviour
- Create a module to export or import data into Joplin
- Define new settings and setting sections, and get/set them from the plugin

5
readme/api/references/development_mode.md Normal file
View File

@@ -0,0 +1,5 @@
# Development mode
When experimenting with Joplin, for example when developing a plugin or trying a theme, you might want to run Joplin in development mode. Doing so means that Joplin will run using a different profile, so you can experiment without risking to accidentally change or delete your data.
To enable Development Mode, open Joplin as normal, then go to **Help => Copy dev mode command to clipboard**. This will copy a command to the clipboard. Now close Joplin, and start it again in dev mode using the command you've just copied.

21
readme/api/references/plugin_manifest.md Normal file
View File

@@ -0,0 +1,21 @@
# Plugin Manifest
The manifest file is a JSON file that describes various properties of the plugin. If you use the Yeoman generator, it should be automatically generated based on the answers you've provided. The supported properties are:
- `manifest_version`: For now should always be "1"
- `name`: Name of the plugin
- `description`: Additional information about the plugin
- `version`: Version number such as "1.0.0"
- `homepage_url`: Homepage URL of the plugin (can also be, for example, a link to a GitHub repository)
Here's a complete example:
```json
{
"manifest_version": 1,
"name": "Joplin Simple Plugin",
"version": "1.0.0",
"description": "To test loading and running a plugin",
"homepage_url": "https://joplinapp.org"
}
```

3
readme/api.md → readme/api/references/rest_api.md
View File

@@ -1,5 +1,6 @@
# Joplin API
# Joplin Data API
This API is available when the clipper server is running. It provides access to the notes, notebooks, tags and other Joplin object via a REST API. Plugins can also access this API even when the clipper server is not running.
In order to use it, you'll first need to find on which port the service is running. To do so, open the Web Clipper Options in Joplin and if the service is running it should tell you on which port. Normally it runs on port **41184**. If you want to find it programmatically, you may follow this kind of algorithm:

341
readme/api/tutorials/toc_plugin.md Normal file
View File

@@ -0,0 +1,341 @@
# Creating a table of content plugin
This tutorial will guide you through the steps to create a table of content plugin for Joplin. It will display a view next to the current note that will contain links to the sections of a note. It will be possible to click on one of the header to jump to the relevant section.
Through this tutorial you will learn about several aspect of the Joplin API, including:
- The plugin API
- How to create a webview
- How to listen to changes in the user interface
## Setting up your environment
Before getting any further, make sure your environment is setup correctly as described in the [Get Started guide](https://github.com/laurent22/joplin/blob/dev/readme/api/get_started/plugins/).
## Registering the plugin
All plugins must [register themselves](https://joplinapp.org/plugins/api/classes/joplinplugins.html) and declare what events they can handle. To do so, open `src/index.ts` and register the plugin as below. We'll also need to run some initialisation code when the plugin starts, so add the `onStart()` event handler too:
```typescript
// Import the Joplin API
import joplin from 'api';
// Register the plugin
joplin.plugins.register({
// Run initialisation code in the onStart event handler
// Note that due to the plugin multi-process architecture, you should
// always assume that all function calls and event handlers are async.
onStart: async function() {
console.info('TOC plugin started!');
},
});
```
If you now build the plugin and try to run it in Joplin, you should see the message `TOC plugin started!` in the dev console.
## Getting the current note
In order to create the table of content, you will need to access the content of the currently selected note, and you will need to refresh the TOC every time the note changes. All this can be done using the [workspace API](https://joplinapp.org/plugins/api/classes/joplinworkspace.html), which provides information about the active content being edited.
So within the `onStart()` event handler, add the following:
```typescript
joplin.plugins.register({
onStart: async function() {
// Later, this is where you'll want to update the TOC
async function updateTocView() {
// Get the current note from the workspace.
const note = await joplin.workspace.selectedNote();
// Keep in mind that it can be `null` if nothing is currently selected!
if (note) {
console.info('Note content has changed! New note is:', note);
} else {
console.info('No note is selected');
}
}
// This event will be triggered when the user selects a different note
await joplin.workspace.onNoteSelectionChange(() => {
updateTocView();
});
// This event will be triggered when the content of the note changes
// as you also want to update the TOC in this case.
await joplin.workspace.onNoteContentChange(() => {
updateTocView();
});
// Also update the TOC when the plugin starts
updateTocView();
},
});
```
Try the above and you should see in the console the event handler being called every time a new note is opened, or whenever the note content changes.
## Getting the note sections and slugs
Now that you have the current note, you'll need to extract the headers from that note in order to build the TOC from it. Since the note content is plain Markdown, there are several ways to do so, such as using a Markdown parser, but for now a quick and dirty solution is to get all the lines that start with any number of `#` followed by a space. Any such line should be a header.
The function below, which you can copy anywhere in your file, will use this method and return an array of headers, with the text and level (H1, H2, etc.) of header:
```typescript
function noteHeaders(noteBody:string) {
const headers = [];
const lines = noteBody.split('\n');
for (const line of lines) {
const match = line.match(/^(#+)\s(.*)*/);
if (!match) continue;
headers.push({
level: match[1].length,
text: match[2],
});
}
return headers;
}
```
Then call this function from your event handler:
```typescript
joplin.plugins.register({
onStart: async function() {
async function updateTocView() {
const note = await joplin.workspace.selectedNote();
if (note) {
const headers = noteHeaders(note.body);
console.info('The note has the following headers', headers);
} else {
console.info('No note is selected');
}
}
// ...
},
});
```
Later you will also need a way to generate the slug for each header. A slug is an identifier which is used to link to a particular header. Essentially a header text like "My Header" is converted to "my-header". And if there's already a slug with that name, a number is appended to it. Without going into too much details, you will need the "slug" package to generate this for you, so install it using `npm i -s slug` from the root of your plugin directory.
Then this is the function you will need for Joplin, so copy it somewhere in your file:
```typescript
const nodeSlug = require('slug');
let slugs = {};
function headerSlug(headerText) {
const s = nodeSlug(headerText);
let num = slugs[s] ? slugs[s] : 1;
const output = [s];
if (num > 1) output.push(num);
slugs[s] = num + 1;
return output.join('-');
}
```
And you will need a utility function to escape HTML. There are many packages to do this but for now you can simply use this:
```typescript
// From https://stackoverflow.com/a/6234804/561309
function escapeHtml(unsafe:string) {
return unsafe
.replace(/&/g, "&")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/'/g, "&#039;");
}
```
Again try to run the plugin and if you select a note with multiple headers, you should see the header list in the console.
## Creating a webview
In order to display the TOC in Joplin, you will need a [webview panel](https://joplinapp.org/plugins/api/classes/joplinviewspanels.html). Panels are a simple way to add custom content to the UI using HTML/CSS and JavaScript. First you would create the panel object and get back a view handler. Using this handler, you can set various properties such as the HTML content.
Here's how it could be done:
```typescript
joplin.plugins.register({
onStart: async function() {
// Create the panel object
const panel = await joplin.views.panels.create();
// Set some initial content while the TOC is being created
await joplin.views.panels.setHtml(panel, 'Loading...');
async function updateTocView() {
const note = await joplin.workspace.selectedNote();
slugs = {}; // Reset the slugs
if (note) {
const headers = noteHeaders(note.body);
// First create the HTML for each header:
const itemHtml = [];
for (const header of headers) {
const slug = headerSlug(header.text);
// - We indent each header based on header.level.
//
// - The slug will be needed later on once we implement clicking on a header.
// We assign it to a "data" attribute, which can then be easily retrieved from JavaScript
//
// - Also make sure you escape the text before inserting it in the HTML to avoid XSS attacks
// and rendering issues. For this use the `escapeHtml()` function you've added earlier.
itemHtml.push(`
<p class="toc-item" style="padding-left:${(header.level - 1) * 15}px">
<a class="toc-item-link" href="#" data-slug="${escapeHtml(slug)}">
${escapeHtml(header.text)}
</a>
</p>
`);
}
// Finally, insert all the headers in a container and set the webview HTML:
await joplin.views.panels.setHtml(panel, `
<div class="container">
${itemHtml.join('\n')}
</div>
`);
} else {
await joplin.views.panels.setHtml(panel, 'Please select a note to view the table of content');
}
}
// ...
},
});
```
Now run the plugin again and you should see the TOC dynamically updating as you change notes.
## Styling the view
In order to better integrate the TOC to Joplin, you might want to style it using CSS. To do so, first add a `webview.css` file next to `index.ts`, then you will need to let Joplin know about this file. This is done using the `addScript()` function (which is also used to add JavaScript files as we'll see later), like so:
```typescript
const panel = await joplin.views.panels.create();
// Add the CSS file to the view, right after it has been created:
await joplin.views.panels.addScript(panel, './webview.css');
```
This file is just a plain CSS file you can use to style your view. Additionally, you can access from there a number of theme variables, which you can use to better integrate the view to the UI. For example, using these variables you can use a dark background in dark mode, and a light one in light mode.
The CSS file below would give the view the correct font color and family, and the right background colour:
```css
/* In webview.css */
.container {
background-color: var(--joplin-background-color);
color: var(--joplin-color);
font-size: var(--joplin-font-size);
font-family: var(--joplin-font-family);
}
.toc-item a {
color: var(--joplin-color);
text-decoration: none;
}
```
Try the plugin and the styling should be improved. You may also try to switch to dark or light mode and see the style being updated.
## Making the webview interactive
The next step is to make the TOC interactive so that when the user clicks on a link, the note is scrolled to right header. This can be done using an external JavaScript file that will handle the click events. As for the CSS file, create a `webview.js` file next to `index.ts`, then add the script to the webview:
```typescript
// In index.ts
const panel = joplin.views.createWebviewPanel();
await joplin.views.panels.addScript(panel, './webview.css');
await joplin.views.panels.addScript(panel, './webview.js'); // Add the JS file
```
To check that everything's working, let's create a simple event handler that display the header slug when clicked:
```javascript
// In webview.js
// There are many ways to listen to click events, you can even use
// something like jQuery or React. This is how it can be done using
// plain JavaScript:
document.addEventListener('click', event => {
const element = event.target;
// If a TOC header has been clicked:
if (element.className === 'toc-item-link') {
// Get the slug and display it:
const slug = element.dataset.slug;
console.info('Clicked header slug: ' + slug);
}
});
```
If everything works well, you should now see the slug in the console whenever you click on a header link. The next step will be to use that slug to scroll to the right header.
## Passing messages between the webview and the plugin
For security reason, webviews run within their own sandbox (iframe) and thus do not have access to the Joplin API. You can however send messages to and from the webview to the plugin, and you can call the Joplin API from the plugin.
From within a webview, you have access to the webviewApi object, which among others has a `postMessage()` function you can use to send a message to the plugin. Let's use this to post the slug info to the plugin:
Change `webview.js` like so:
```javascript
document.addEventListener('click', event => {
const element = event.target;
if (element.className === 'toc-item-link') {
// Post the message and slug info back to the plugin:
webviewApi.postMessage({
name: 'scrollToHash',
hash: element.dataset.slug,
});
}
});
```
Then from the plugin, in `src/index.ts`, you can listen to this message using the `onMessage()` handler. Then from this handler, you can call the `scrollToHash` command and pass it the slug (or hash).
```typescript
joplin.plugins.register({
onStart: async function() {
const panel = await joplin.views.panels.create();
// ...
await joplin.views.panels.onMessage(panel, (message) => {
if (message.name === 'scrollToHash') {
// As the name says, the scrollToHash command makes the note scroll
// to the provided hash.
joplin.commands.execute('scrollToHash', {
hash: message.hash,
})
}
});
// ...
}
// ...
```
And that's it! If you run this code you should now have a fully functional TOC. The full source code is available there:
https://github.com/laurent22/joplin/tree/dev/CliClient/tests/support/plugins/toc/
Various improvements can be made such as improving the styling, making the header collapsible, etc. but that tutorial should provide the basic building blocks to do so. You might also want to check the [plugin API](https://joplinapp.org/plugins/api/classes/joplin.html) for further information or head to the [development forum](https://discourse.joplinapp.org/c/development/6) for support.

2
readme/desktop.md
View File

@@ -1,3 +1,5 @@
# Desktop application
<img src="https://joplinapp.org/images/DemoDesktop.png" style="max-width: 100%">
For general information relevant to all the applications, see also [Joplin home page](https://joplinapp.org).

30
readme/faq.md
View File

@@ -1,4 +1,6 @@
# Installer gets stuck on Windows
# FAQ
## Installer gets stuck on Windows
The installer may get stuck if the app was not uninstalled correctly. To fix the issue you will need to clean up the left-over entry from the Registry. To do so please follow these steps:
@@ -12,7 +14,7 @@ Now try to install again and it should work.
More info there: https://github.com/electron-userland/electron-builder/issues/4057
# How can I edit my note in an external text editor?
## How can I edit my note in an external text editor?
The editor command (may include arguments) defines which editor will be used to open a note. If none is provided it will try to auto-detect the default editor. If this does nothing or you want to change it for Joplin, you need to configure it in the Preferences -> Text editor command.
@@ -41,11 +43,11 @@ notepad++.exe --openSession # Opens Notepad ++ in new window
Note that the path to directory with your editor executable must exist in your PATH variable ([Windows](https://www.computerhope.com/issues/ch000549.htm), [Linux/Mac](https://opensource.com/article/17/6/set-path-linux)) If not, the full path to the executable must be provided.
# When I open a note in vim, the cursor is not visible
## When I open a note in vim, the cursor is not visible
It seems to be due to the setting `set term=ansi` in .vimrc. Removing it should fix the issue. See https://github.com/laurent22/joplin/issues/147 for more information.
# All my notes got deleted after changing the WebDAV URL!
## All my notes got deleted after changing the WebDAV URL!
When changing the WebDAV URL, make sure that the new location has the same exact content as the old location (i.e. copy all the Joplin data over to the new location). Otherwise, if there's nothing on the new location, Joplin is going to think that you have deleted all your data and will proceed to delete it locally too. So to change the WebDAV URL, please follow these steps:
@@ -57,19 +59,19 @@ When changing the WebDAV URL, make sure that the new location has the same exact
6. Synchronise to verify that everything is working.
7. Do step 5 and 6 for all the other Joplin clients you need to sync.
# How can I easily enter Markdown tags in Android?
## How can I easily enter Markdown tags in Android?
You may use a special keyboard such as [Multiling O Keyboard](https://play.google.com/store/apps/details?id=kl.ime.oh&hl=en), which has shortcuts to create Markdown tags. [More information in this post](https://discourse.joplinapp.org/t/android-create-new-list-item-with-enter/585/2?u=laurent).
# The initial sync is very slow, how can I speed it up?
## The initial sync is very slow, how can I speed it up?
Whenever importing a large number of notes, for example from Evernote, it may take a very long time for the first sync to complete. There are various techniques to speed thing up (if you don't want to simply wait for the sync to complete), which are outlined in [this post](https://discourse.joplinapp.org/t/workaround-for-slow-initial-bulk-sync-after-evernote-import/746?u=laurent).
# Is it possible to use real file and folder names in the sync target?
## Is it possible to use real file and folder names in the sync target?
Unfortunately it is not possible. Joplin synchronises with file systems using an open format however it does not mean the sync files are meant to be user-editable. The format is designed to be performant and reliable, not user friendly (it cannot be both), and that cannot be changed. Joplin sync directory is basically just a database.
# Could there be a password to restrict access to Joplin?
## Could there be a password to restrict access to Joplin?
The end to end encryption that Joplin implements is to protect the data during transmission and on the cloud service so that only you can access it.
@@ -79,9 +81,9 @@ For these reasons, because the OS or yourself can easily protect the local data,
There is however an issue open about it, so pull requests are welcome: https://github.com/laurent22/joplin/issues/289
# WebDAV synchronisation is not working
## WebDAV synchronisation is not working
## "Forbidden" error in Strato
### "Forbidden" error in Strato
For example:
@@ -96,17 +98,17 @@ For example:
In this case, [make sure you enter the correct WebDAV URL](https://github.com/laurent22/joplin/issues/309).
## Nextcloud sync is not working
### Nextcloud sync is not working
- Check your username and password. **Type it manually** (without copying and pasting it) and try again.
- Check the WebDAV URL - to get the correct URL, go to Nextcloud and, in the left sidebar, click on "Settings" and copy the WebDAV URL from there. **Do not forget to add the folder you've created to that URL**. For example, if the base the WebDAV URL is "https://example.com/nextcloud/remote.php/webdav/" and you want the notes to be synced in the "Joplin" directory, you need to give the URL "https://example.com/nextcloud/remote.php/webdav/Joplin" **and you need to create the "Joplin" directory yourself**.
- Did you enable **2FA** (Multi-factor authentication) on Nextcloud? In that case, you need to [create a one-time password for Joplin on the Nextcloud admin interface](https://github.com/laurent22/joplin/issues/1453#issuecomment-486640902).
# How can I use self-signed SSL certificates on Android?
## How can I use self-signed SSL certificates on Android?
If you want to serve using https but can't or don't want to use SSL certificates signed by trusted certificate authorities (like "Let's Encrypt"), it's possible to generate a custom CA and sign your certificates with it. You can generate the CA and certificates using [openssl](https://gist.github.com/fntlnz/cf14feb5a46b2eda428e000157447309), but I like to use a tool called [mkcert](https://github.com/FiloSottile/mkcert) for it's simplicity. Finally, you have to add your CA certificate to Android settings so that Android can recognize the certificates you signed with your CA as valid ([link](https://support.google.com/nexus/answer/2844832?hl=en-GB)).
# How do I restart Joplin on Windows (so that certain changes take effect)?
## How do I restart Joplin on Windows (so that certain changes take effect)?
If `Show tray icon` is enabled, closing the Joplin window does not quit the application. To restart the application properly, one of the following has to be done to quit Joplin:
@@ -115,6 +117,6 @@ If `Show tray icon` is enabled, closing the Joplin window does not quit the appl
Additionally the Windows Task Manager can be used to verify whether Joplin is still around.
# Why is it named Joplin?
## Why is it named Joplin?
The name comes from the composer and pianist [Scott Joplin](https://en.wikipedia.org/wiki/Scott_Joplin), which I often listen to. His name is also easy to remember and type so it felt like a good choice. And, to quote a user on Hacker News, "though Scott Joplin's ragtime musical style has a lot in common with some very informal music, his own approach was more educated, sophisticated, and precise. Every note was in its place for a reason, and he was known to prefer his pieces to be performed exactly as written. So you could say that compared to the people who came before him, his notes were more organized".

3
readme/feedback.md
View File

@@ -1,3 +0,0 @@
https://help.nextcloud.com/t/mobile-note-taking-with-your-private-cloud-announcing-joplin-nextcloud-integration/29239/39?u=laurent
We are a small Church-Office in Germany with several employees and members of staff who have little or no affinity to working with computer. [...] There is special information of a very high intimacy and confidentiality level that should only be accessible to the owner of the information, e.g. notes about counseling. And here Joplin comes into action. When used with encryption, the notes can only be read by the owner, not even by our cloud operator. With Joplin I can guaranty my users the highest level of confidentiality possible, combined with a clear and easy to use interface.

2
readme/gsoc2020/ideas.md
View File

@@ -1,3 +1,5 @@
# GSoC 2020 Ideas
2020 is Joplin first round at Google Summer of Code. Detailed information on how to get involved and apply are given in the [general Summer of Code introduction](https://joplinapp.org/gsoc2020/)
**These are all proposals! We are open to new ideas you might have!!** Do you have an awesome idea you want to work on with Joplin but that is not among the ideas below? That's cool. We love that! But please do us a favour: Get in touch with a mentor early on and make sure your project is realistic and within the scope of Joplin.

2
readme/mobile.md
View File

@@ -1 +1,3 @@
# Mobile app
An Android and iOS (iPhone/iPad) applications are available from the [Joplin home page](https://joplinapp.org).

7
readme/plugins.md Normal file
View File

@@ -0,0 +1,7 @@
# Plugins
Joplin supports plugins, which can be used to add new features or modify the application behaviour.
## Installing a plugin
To install a plugin, copy its directory to your profile's `plugins` directory. The plugin will be automatically loaded and executed when you restart the application.

67
readme/spec.md
View File

@@ -1,67 +0,0 @@
# Encryption
Encrypted data is encoded to ASCII because encryption/decryption functions in React Native can only deal with strings. So for compatibility with all the apps we need to use the lowest common denominator.
## Encrypted data format
### Header
Name | Size
-------------------|-------------------------
Identifier | 3 chars ("JED")
Version number | 2 chars (Hexa string)
This is followed by the encryption metadata:
Name | Size
-------------------|-------------------------
Length | 6 chars (Hexa string)
Encryption method | 2 chars (Hexa string)
Master key ID | 32 chars (Hexa string)
See `lib/services/EncryptionService.js` for the list of available encryption methods.
### Data chunk
The data is encoded in one or more chunks for performance reasons. That way it is possible to take a block of data from one file and encrypt it to another block in another file. Encrypting/decrypting the whole file in one go would not work (on mobile especially).
Name | Size
--------|----------------------------
Length | 6 chars (Hexa string)
Data | ("Length" bytes) (ASCII)
## Master Keys
The master keys are used to encrypt and decrypt data. They can be generated from the Encryption Service and are saved to the database. They are themselves encrypted via a user password using a [strong encryption method](https://github.com/laurent22/joplin/blob/fb6dee32ac035b00153106273135fb16be4b4fa5/ReactNativeClient/lib/services/EncryptionService.js#L263).
These encrypted master keys are transmitted with the sync data so that they can be available to each client. Each client will need to supply the user password to decrypt each key.
The application supports multiple master keys in order to handle cases where one offline client starts encrypting notes, then another offline client starts encrypting notes too, and later both sync. Both master keys will have to be decrypted separately with the user password.
Only one master key can be active for encryption purposes. For decryption, the algorithm will check the Master Key ID in the header, then check if it's available to the current app and, if so, use this for decryption.
## Encryption Service
The applications make use of the `EncryptionService` class to handle encryption and decryption. Before it can be used, a least one master key must be loaded into it and be marked as "active".
## Encryption workflow
Items are encrypted only during synchronisation, when they are serialised (via `BaseItem.serializeForSync`), so before being sent to the sync target.
They are decrypted by DecryptionWorker in the background.
The apps handle displaying both decrypted and encrypted items, so that user is aware that these items are there even if not yet decrypted. Encrypted items are mostly read-only to the user, except that they can be deleted.
## Enabling and disabling encryption
Enabling/disabling E2EE while two clients are in sync might have an unintuitive behaviour (although that behaviour might be correct), so below some scenarios are explained:
- If client 1 enables E2EE, all items will be synced to target and will appear encrypted on target. Although all items have been re-uploaded to the target, their timestamps did *not* change (because the item data itself has not changed, only its representation). Because of this, client 2 will not re-download the items - it does not need to do so anyway since it has already the item data.
- When a client sync and download a master key for the first time, encryption will be automatically enabled (user will need to supply the master key password). In that case, all items that are not encrypted will be re-synced. Uploading only non-encrypted items is an optimisation since if an item is already encrypted locally it means it's encrypted on target too.
- If both clients are in sync with E2EE enabled: if client 1 disable E2EE, it's going to re-upload all the items unencrypted. Client 2 again will not re-download the items for the same reason as above (data did not change, only representation). Note that user *must* manually disable E2EE on all clients otherwise some will continue to upload encrypted items. Since synchronisation is stateless, clients do not know whether other clients use E2EE or not so this step has to be manual.
- Although messy, Joplin supports having some clients send encrypted items and others unencrypted ones. The situation gets resolved once all the clients have the same E2EE settings.
- Currently, there is no way to delete encryption keys if you do not need them anymore or if you disabled the encryption completely. You will get a persistant notification to provide a Master Key password on a new device, even if encryption is disabled. Entering the Master Key(s) password and still having the encryption disabled will get rid of the notification. See [Delete E2EE Master Keys](https://discourse.joplinapp.org/t/delete-e2ee-master-keys/906) for more info.

67
readme/spec/e2ee.md
View File

@@ -0,0 +1,67 @@
# Encryption
Encrypted data is encoded to ASCII because encryption/decryption functions in React Native can only deal with strings. So for compatibility with all the apps we need to use the lowest common denominator.
## Encrypted data format
### Header
Name | Size
-------------------|-------------------------
Identifier | 3 chars ("JED")
Version number | 2 chars (Hexa string)
This is followed by the encryption metadata:
Name | Size
-------------------|-------------------------
Length | 6 chars (Hexa string)
Encryption method | 2 chars (Hexa string)
Master key ID | 32 chars (Hexa string)
See `lib/services/EncryptionService.js` for the list of available encryption methods.
### Data chunk
The data is encoded in one or more chunks for performance reasons. That way it is possible to take a block of data from one file and encrypt it to another block in another file. Encrypting/decrypting the whole file in one go would not work (on mobile especially).
Name | Size
--------|----------------------------
Length | 6 chars (Hexa string)
Data | ("Length" bytes) (ASCII)
## Master Keys
The master keys are used to encrypt and decrypt data. They can be generated from the Encryption Service and are saved to the database. They are themselves encrypted via a user password using a [strong encryption method](https://github.com/laurent22/joplin/blob/fb6dee32ac035b00153106273135fb16be4b4fa5/ReactNativeClient/lib/services/EncryptionService.js#L263).
These encrypted master keys are transmitted with the sync data so that they can be available to each client. Each client will need to supply the user password to decrypt each key.
The application supports multiple master keys in order to handle cases where one offline client starts encrypting notes, then another offline client starts encrypting notes too, and later both sync. Both master keys will have to be decrypted separately with the user password.
Only one master key can be active for encryption purposes. For decryption, the algorithm will check the Master Key ID in the header, then check if it's available to the current app and, if so, use this for decryption.
## Encryption Service
The applications make use of the `EncryptionService` class to handle encryption and decryption. Before it can be used, a least one master key must be loaded into it and be marked as "active".
## Encryption workflow
Items are encrypted only during synchronisation, when they are serialised (via `BaseItem.serializeForSync`), so before being sent to the sync target.
They are decrypted by DecryptionWorker in the background.
The apps handle displaying both decrypted and encrypted items, so that user is aware that these items are there even if not yet decrypted. Encrypted items are mostly read-only to the user, except that they can be deleted.
## Enabling and disabling encryption
Enabling/disabling E2EE while two clients are in sync might have an unintuitive behaviour (although that behaviour might be correct), so below some scenarios are explained:
- If client 1 enables E2EE, all items will be synced to target and will appear encrypted on target. Although all items have been re-uploaded to the target, their timestamps did *not* change (because the item data itself has not changed, only its representation). Because of this, client 2 will not re-download the items - it does not need to do so anyway since it has already the item data.
- When a client sync and download a master key for the first time, encryption will be automatically enabled (user will need to supply the master key password). In that case, all items that are not encrypted will be re-synced. Uploading only non-encrypted items is an optimisation since if an item is already encrypted locally it means it's encrypted on target too.
- If both clients are in sync with E2EE enabled: if client 1 disable E2EE, it's going to re-upload all the items unencrypted. Client 2 again will not re-download the items for the same reason as above (data did not change, only representation). Note that user *must* manually disable E2EE on all clients otherwise some will continue to upload encrypted items. Since synchronisation is stateless, clients do not know whether other clients use E2EE or not so this step has to be manual.
- Although messy, Joplin supports having some clients send encrypted items and others unencrypted ones. The situation gets resolved once all the clients have the same E2EE settings.
- Currently, there is no way to delete encryption keys if you do not need them anymore or if you disabled the encryption completely. You will get a persistant notification to provide a Master Key password on a new device, even if encryption is disabled. Entering the Master Key(s) password and still having the encryption disabled will get rid of the notification. See [Delete E2EE Master Keys](https://discourse.joplinapp.org/t/delete-e2ee-master-keys/906) for more info.

103
readme/spec/plugins.md Normal file
View File

@@ -0,0 +1,103 @@
# Plugin system architecture
The plugin system assumes a multi-process architecture, which is safer and easier to manage. For example if a plugin freezes or crashes, it doesn't bring down the app with it. It also makes it easier to find the source of problem when there is one - eg. we know that process X has crashed so the problem is with the plugin running inside. The alternative, to run everything within the same process, would make it very hard to make such a diagnostic. Once a plugin call is frozen in an infinite loop or crashes the app, we can't know anything.
## Main architecture elements
### Plugin script
Written by the user and loaded by Joplin, it's a simple JavaScript file that makes calls to the plugin API. It is loaded in a separate process.
### Sandbox proxy
It is loaded in the same process as the plugin script. Whenever the plugin script calls a plugin API function (eg. joplin.commands.execute) it goes through this proxy. The proxy then converts the call to a plain string and use IPC to send the call to the plugin host. The plugin host executes the function on the plugin API then sends back the result by IPC call again.
### Plugin host
The plugin host is simply the main application. Its role is to start and initialise the plugin service and to load plugins from the provided script files.
### Plugin service
It is used to load and run plugins. Running plugins is platform-specific, thus this part is injected into the service via a platform-specific Plugin Runner.
### Plugin runner
This is the platform-specfic way to load and run a plugin. For example, on desktop, it creates a new BrowserWindow (which is a new process), then load the script inside. On Cli, for now the "vm" package is used, so the plugin actually runs within the same process.
The plugin runner also initialises the sandbox proxy and injects it into the plugin code.
### Plugin API
The plugin API is a light wrapper over Joplin's internal functions and services. All the platforms share some of the plugin API but there can also be some differences. For example, the desktop app exposes the text editor component commands, and so this part of the plugin API is available only on desktop. The difference between platforms is implemented using the PlatformImplementation class, which is injected in the plugin service on startup.
## Handling events between the plugin and the host
Handling events in plugins is relatively complicated due to the need to send IPC messages and the limitations of the IPC protocol, which in particular cannot transfer functions.
For example, let's say we define a command in the plugin:
```typescript
joplin.commands.register({
name: 'testCommand1',
label: 'My Test Command 1',
}, {
onExecute: (args:any) => {
alert('Testing plugin command 1');
},
});
```
The "onExecute" event handler needs to be called whenever, for example, a toolbar button associated with this command is clicked. The problem is that it is not possible to send a function via IPC (which can only transfer plain objects), so there has to be a translation layer in between.
The way it is done in Joplin is like so:
In the **sandbox proxy**, the event handlers are converted to string event IDs and the original event handler is stored in a map before being sent to host via IPC. So in the example above, the command would be converted to this plain object:
```typescript
{
name: 'testCommand1',
label: 'My Test Command 1',
}, {
onExecute: '___event_handler_123',
}
```
Then, still in the sandbox proxy, we'll have a map called something like `eventHandlers`, which now will have this content:
```typescript
eventHandlers['___event_handler_123'] = (args:any) => {
alert('Testing plugin command 1');
}
```
In the **plugin runner** (Host side), all the event IDs are converted to functions again, but instead of performing the action directly, it posts an IPC message back to the sandbox proxy using the provided event ID.
So in the host, the command will now look like this:
```typescript
{
name: 'testCommand1',
label: 'My Test Command 1',
}, {
onExecute: (args:any) => {
postMessage('pluginMessage', { eventId: '___event_handler_123', args: args });
};
}
```
At this point, any code in the Joplin application can call the `onExecute` function as normal without having to know about the IPC translation layer.
When the function onExecute is eventually called, the IPC message is sent back to the sandbox proxy, which will decode it and execute it.
So on the **sandbox proxy**, we'll have something like this:
```typescript
window.addEventListener('message', ((event) => {
const eventId = getEventId(event); // Get back the event ID (implementation might be different)
const eventArgs = getEventArgs(event); // Get back the args (implementation might be different)
if (eventId) {
// And call the event handler
eventHandlers[eventId](...eventArgs);
}
}));
```