Merge #2631

2631: Restful api finishing touches r=mergify[bot] a=Diman0

## What type of PR?

enhancement

## What does this PR do?
Some finishing touches for the restful api.

- Make the API configurable via the setup utility.  
  - Configured exactly the same as the ADMIN and WEBMAIL. 
- We have a single config (API) that configures whether it is exposed (via front). Just like ADMIN. The API is always reachable by directly connecting to the admin container.
- API_TOKEN does not enable/disable the API anymore. When it is not configured, an error is returned (via the internet browser) that the API_TOKEN must be configured in mailu.env.
- Fix some small bugs in the setup utility ( selecting none in the dropdown boxes, now correctly changes the config)
- Update Flask-RestX to 1.0.5. This resolves the deprecation warnings introduced by Flask-RestX.

### Related issue(s)

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>

core/admin/mailu/__init__.py

@@ -74,8 +74,7 @@ def create_app_from_config(config):
     app.register_blueprint(ui.ui, url_prefix=app.config['WEB_ADMIN'])
     app.register_blueprint(internal.internal, url_prefix='/internal')
     app.register_blueprint(sso.sso, url_prefix='/sso')
-    if app.config.get('API_TOKEN'):
+    api.register(app, web_api_root=app.config.get('WEB_API'))
-        api.register(app, web_api_root=app.config.get('WEB_API'))
     return app
 
 

core/admin/mailu/api/__init__.py

@@ -8,17 +8,25 @@ def register(app, web_api_root):
     # register api bluprint(s)
     apidoc.apidoc.url_prefix = f'{web_api_root}/v{int(APIv1.VERSION)}'
     APIv1.api_token = app.config['API_TOKEN']
-    app.register_blueprint(APIv1.blueprint, url_prefix=f'{web_api_root}/v{int(APIv1.VERSION)}')
+    if app.config['API_TOKEN'] != '':
+        app.register_blueprint(APIv1.blueprint, url_prefix=f'{web_api_root}/v{int(APIv1.VERSION)}')
 
-    # add redirect to current api version
+        # add redirect to current api version
-    redirect_api = Blueprint('redirect_api', __name__)
+        redirect_api = Blueprint('redirect_api', __name__)
-    @redirect_api.route('/')
+        @redirect_api.route('/')
-    def redir():
+        def redir():
-        return redirect(url_for(f'{APIv1.blueprint.name}.root'))
+            return redirect(url_for(f'{APIv1.blueprint.name}.root'))
-    app.register_blueprint(redirect_api, url_prefix=f'{web_api_root}')
+        app.register_blueprint(redirect_api, url_prefix=f'{web_api_root}')
 
-    # swagger ui config
+        # swagger ui config
-    app.config.SWAGGER_UI_DOC_EXPANSION = 'list'
+        app.config.SWAGGER_UI_DOC_EXPANSION = 'list'
-    app.config.SWAGGER_UI_OPERATION_ID = True
+        app.config.SWAGGER_UI_OPERATION_ID = True
-    app.config.SWAGGER_UI_REQUEST_DURATION = True
+        app.config.SWAGGER_UI_REQUEST_DURATION = True
-    app.config.RESTX_MASK_SWAGGER = False
+        app.config.RESTX_MASK_SWAGGER = False
+    else:
+        api = Blueprint('api', __name__)
+        @api.route('/', defaults={'path': ''})
+        @api.route('/<path:path>')
+        def api_token_missing(path):
+            return "<p>Error: API_TOKEN is not configured</p>", 500
+        app.register_blueprint(api, url_prefix=f'{web_api_root}')

core/base/requirements-prod.txt

@@ -27,7 +27,7 @@ Flask-DebugToolbar==0.13.1
 Flask-Login==0.6.2
 flask-marshmallow==0.14.0
 Flask-Migrate==3.1.0
-Flask-RESTX==1.0.3
+Flask-RESTX==1.0.5
 Flask-SQLAlchemy==2.5.1
 Flask-WTF==1.0.1
 frozenlist==1.3.1

docs/api.rst

@@ -5,21 +5,22 @@ Mailu offers a RESTful API for changing the Mailu configuration.
 Anything that can be configured via the Mailu web administration interface,
 can also be configured via the API.
 
-The Mailu API is disabled by default. It can be enabled and configured via
+The Mailu API can be configured via the setup utility (setup.mailu.io).
-the settings:
+It can also be manually configured via mailu.env:
 
-* ``API``
+* ``API`` - Expose the API interface (value: true, false)
-* ``WEB_API``
+* ``WEB_API`` - Path to the API interface
-* ``API_TOKEN``
+* ``API_TOKEN`` - API token for authentication
 
-For more information see the section :ref:`Advanced configuration <advanced_settings>`
+For more information refer to the detailed descriptions in the
-in the configuration reference.
+:ref:`configuration reference <advanced_settings>`.
 
 
 Swagger.json
 ------------
 
-The swagger.json file can be retrieved via: https://myserver/api/v1/swagger.json.
+The swagger.json file can be retrieved via: https://myserver/api/v1/swagger.json
+(WEB_API=/api)
 The swagger.json file can be consumed in programs such as Postman for generating all API calls.
 
 
@@ -28,4 +29,5 @@ In-built SwaggerUI
 The Mailu API comes with an in-built SwaggerUI. It is a web client that allows
 anyone to visualize and interact with the Mailu API.
 
-It is accessible via the URL: https://myserver/api/v1/swaggerui
+Assuming ``/api`` is configured as value for ``WEB_API``, it
+is accessible via the URL: https://myserver/api/

docs/configuration.rst

@@ -141,13 +141,15 @@ Web settings
 
 - ``WEB_WEBMAIL`` contains the path to the Web email client.
 
+- ``WEB_API`` contains the path to the RESTful API.
+
 - ``WEBROOT_REDIRECT`` redirects all non-found queries to the set path.
   An empty ``WEBROOT_REDIRECT`` value disables redirecting and enables
   classic behavior of a 404 result when not found.
   Alternatively, ``WEBROOT_REDIRECT`` can be set to ``none`` if you
   are using an Nginx override for ``location /``.
 
-All three options need a leading slash (``/``) to work.
+All four options need a leading slash (``/``) to work.
 
   .. note:: ``WEBROOT_REDIRECT`` has to point to a valid path on the webserver.
     This means it cannot point to any services which are not enabled.
@@ -204,13 +206,9 @@ Depending on your particular deployment you most probably will want to change th
 Advanced settings
 -----------------
 
-The ``API`` (default: False) setting controls if the API endpoint is reachable.
+The ``API_TOKEN`` (default: None) configures the authentication token.
-
+This token must be passed as request header to the API as authentication token.
-The ``WEB_API`` (default: /api) setting configures the endpoint that the API
+This is a mandatory setting for using the RESTful API.
-listens on publicly&interally. The path must always start with a leading slash.
-
-The ``API_TOKEN`` (default: None) enables the API endpoint. This token must be
-passed as request header to the API as authentication token.
 
 The ``CREDENTIAL_ROUNDS`` (default: 12) setting is the number of rounds used by the
 password hashing scheme. The number of rounds can be reduced in case faster

setup/flavors/compose/mailu.env

@@ -52,6 +52,9 @@ ADMIN={{ admin_enabled or 'false' }}
 # Choose which webmail to run if any (values: roundcube, snappymail, none)
 WEBMAIL={{ webmail_type }}
 
+# Expose the API interface (value: true, false)
+API={{ api_enabled or 'false' }}
+
 # Dav server implementation (value: radicale, none)
 WEBDAV={{ webdav_enabled or 'none' }}
 
@@ -131,6 +134,9 @@ WEB_WEBMAIL=/
 WEB_WEBMAIL={{ webmail_path }}
 {% endif %}
 
+# Path to the API interface if enabled
+WEB_API={{ api_path }}
+
 # Website name
 SITENAME={{ site_name }}
 
@@ -182,6 +188,10 @@ TZ=Etc/UTC
 # Default spam threshold used for new users
 DEFAULT_SPAM_THRESHOLD=80
 
+# API token required for authenticating to the RESTful API.
+# This is a mandatory setting for using the RESTful API.
+API_TOKEN={{ api_token }}
+
 ###################################
 # Database settings
 ###################################

setup/static/render.js

@@ -1,18 +1,28 @@
+//API_TOKEN generator
+var chars = "0123456789abcdefghijklmnopqrstuvwxyz!@#$%^&*()ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+var tokenLength = 12;
+var token = "";
+
+for (var i = 0; i <= tokenLength; i++) {
+	var randomNumber = Math.floor(Math.random() * chars.length);
+	token += chars.substring(randomNumber, randomNumber +1);
+   }
+
 $(document).ready(function() {
 	if ($("#webmail").val() == 'none') {
 		$("#webmail_path").hide();
-		$("#webmail_path").attr("value", "");
+		$("#webmail_path").val("");
 	} else {
 		$("#webmail_path").show();
-		$("#webmail_path").attr("value", "/webmail");
+		$("#webmail_path").val("/webmail");
 	}
 	$("#webmail").click(function() {
 		if (this.value == 'none') {
 			$("#webmail_path").hide();
-			$("#webmail_path").attr("value", "");
+			$("#webmail_path").val("");
 		} else {
 			$("#webmail_path").show();
-			$("#webmail_path").attr("value", "/webmail");
+			$("#webmail_path").val("/webmail");
 		}
 	});
 });
@@ -20,15 +30,50 @@ $(document).ready(function() {
 $(document).ready(function() {
 	if ($('#admin').prop('checked')) {
 		$("#admin_path").show();
-		$("#admin_path").attr("value", "/admin");
+		$("#admin_path").val("/admin");
 	}
 	$("#admin").change(function() {
 		if ($(this).is(":checked")) {
 			$("#admin_path").show();
-			$("#admin_path").attr("value", "/admin");
+			$("#admin_path").val("/admin");
 		} else {
 			$("#admin_path").hide();
-			$("#admin_path").attr("value", "");
+			$("#admin_path").val("");
+		}
+	});
+});
+
+$(document).ready(function() {
+	if ($('#api').prop('checked')) {
+		$("#api_path").show();
+		$("#api_path").val("/api")
+		$("#api_token").show();
+		$("#api_token").prop('required',true);
+		$("#api_token").val(token);
+		$("#api_token_label").show();
+	} else {
+		$("#api_path").hide();
+		$("#api_path").val("/api")
+		$("#api_token").hide();
+		$("#api_token").prop('required',false);
+		$("#api_token").val("");
+		$("#api_token_label").hide();
+	}
+	$("#api").change(function() {
+		if ($(this).is(":checked")) {
+			$("#api_path").show();
+			$("#api_path").val("/api");
+			$("#api_token").show();
+			$("#api_token").prop('required',true);
+			$("#api_token").val(token)
+			$("#api_token_label").show();
+		} else {
+			$("#api_path").hide();
+			$("#api_path").val("/api")
+			$("#api_token").hide();
+			$("#api_token").prop('required',false);
+			$("#api_token").val("");
+			$("#api_token_label").hide();
 		}
 	});
 });

setup/templates/steps/config.html

@@ -87,6 +87,19 @@ manage your email domains, users, etc.</p>
     <input class="form-control" type="text" name="admin_path" id="admin_path" style="display: none">
 </div>
 
+<p>The API interface is a RESTful API for changing the Mailu configuration.
+  Anything that can be configured via the Mailu web administration interface,
+  can also be configured via the RESTful API. For enabling the API, an API token must be configured.
+  It is not possible to use the API without an API token.</p>
+
+  <div class="form-group">
+      <input type="checkbox" name="api_enabled" value="false" id="api" >
+      <label>Enable the API (and path to the API)</label>
+      <input class="form-control" type="text" name="api_path" id="api_path" style="display: none">
+      <label name="api_token_label" id="api_token_label">API token</label>
+      <input class="form-control" type="text" name="api_token" id="api_token" style="display: none">
+  </div>
+
 <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
 
 <script type="text/javascript" src="{{ url_for('static', filename='render.js') }}"></script>