self-hosted/Mailu
1
0
Fork 0
mirror of https://github.com/Mailu/Mailu.git synced 2025-08-10 22:31:47 +02:00
Code Issues Releases Activity

Introduce connection string (database url) for roundcube.

Browse Source 
Remove database choice from setup.
Remove the old *DB_* database env variables from the documentation.
The env vars are deprecated now. They will be removed after the upcoming
Mailu release.
This commit is contained in:
Dimitri Huisman
2023-03-26 12:21:00 +00:00
parent 6f3a01e31c
commit 709edb522b
10 changed files with 47 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.github/workflows/build_test_deploy.yml vendored
View File

@@ -154,6 +154,7 @@ jobs:
shell: bash
command: |
set -euxo pipefail \
; /usr/bin/docker info \
; echo "${{ github.token }}" | docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
; echo "$DOCKER_PASSW" | docker login --username "$DOCKER_LOGIN" --password-stdin \
; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
@@ -220,6 +221,7 @@ jobs:
shell: bash
command: |
set -euxo pipefail \
; /usr/bin/docker info \
; echo "${{ github.token }}" | docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
; echo "$DOCKER_PASSW2" | docker login --username "$DOCKER_LOGIN2" --password-stdin \
; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
@@ -295,6 +297,7 @@ jobs:
shell: bash
command: |
set -euxo pipefail \
; /usr/bin/docker info \
; echo "${{ github.token }}" | docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
; echo "$DOCKER_PASSW" | docker login --username "$DOCKER_LOGIN" --password-stdin \
; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
@@ -368,6 +371,7 @@ jobs:
shell: bash
command: |
set -euxo pipefail \
; /usr/bin/docker info \
; echo "${{ github.token }}" | docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
; echo "$DOCKER_PASSW2" | docker login --username "$DOCKER_LOGIN2" --password-stdin \
; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \

1
core/admin/mailu/configuration.py
View File

@@ -27,6 +27,7 @@ DEFAULT_CONFIG = {
'DB_NAME': 'mailu',
'SQLITE_DATABASE_FILE': 'data/main.db',
'SQLALCHEMY_DATABASE_URI': 'sqlite:////data/main.db',
'SQLALCHEMY_DATABASE_URI_ROUNDCUBE': 'sqlite:////data/roundcube.db',
'SQLALCHEMY_TRACK_MODIFICATIONS': False,
# Statistics management
'INSTANCE_ID_PATH': '/data/instance',

2
core/base/requirements-prod.txt
View File

@@ -45,7 +45,7 @@ marshmallow==3.18.0
marshmallow-sqlalchemy==0.28.1
msoffcrypto-tool==5.0.0
multidict==6.0.2
mysql-connector-python==8.0.29
mysql-connector-python==8.0.32
olefile==0.46
oletools==0.60.1
packaging==21.3

26
docs/configuration.rst
View File

@@ -285,26 +285,22 @@ These are used for DNS based service discovery with possibly changing services I
Database settings
-----------------
The admin service stores configurations in a database. The roundcube service also stores configurations in a database.
By default SQLite is used. SQLite is sufficient for practically any deployment.
The database contains static configuration. It is recommended to use SQLite. However, a different database back-end can be used via a
`DB URL`_. For Admin, only supported are SQLite, PostgreSQL and MariaDB/MySQL.
The admin service stores configurations in a database.
- ``SQLALCHEMY_DATABASE_URI`` (default: sqlite:////data/main.db): the SQLAlchemy database URL for accessing the database
- ``SQLALCHEMY_DATABASE_URI_ROUNDCUBE`` (default: sqlite:////data/roundcube.db): the Roundcube database URL for accessing the Roundcube database
- ``DB_FLAVOR``: the database type for mailu admin service. (``sqlite``, ``postgresql``, ``mysql``)
- ``DB_HOST``: the database host for mailu admin service. For non-default ports use the notation `host:port`. (when not ``sqlite``)
- ``DB_PW``: the database password for mailu admin service. (when not ``sqlite``)
- ``DB_USER``: the database user for mailu admin service. (when not ``sqlite``)
- ``DB_NAME``: the database name for mailu admin service. (when not ``sqlite``)
For PostgreSQL use driver postgresql (``SQLALCHEMY_DATABASE_URI=postgresql://mailu:mailu_secret_password@database/mailu``).
Alternatively, if you need more control, you can use a `DB URL`_ : do not set any of the ``DB_`` settings and set ``SQLALCHEMY_DATABASE_URI`` instead.
For MariaDB/MySQL use driver mysql+mysqlconnector (``SQLALCHEMY_DATABASE_URI= mysql+mysqlconnector://mailu:mailu_secret_password@database/mailu```).
For Roundcube, refer to the `roundcube documentation`_ for the URL specification.
.. _`DB URL`: https://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls
The roundcube service stores configurations in a database.
- ``ROUNDCUBE_DB_FLAVOR``: the database type for roundcube service. (``sqlite``, ``postgresql``, ``mysql``)
- ``ROUNDCUBE_DB_HOST``: the database host for roundcube service. For non-default ports use the notation `host:port`. (when not ``sqlite``)
- ``ROUNDCUBE_DB_PW``: the database password for roundcube service. (when not ``sqlite``)
- ``ROUNDCUBE_DB_USER``: the database user for roundcube service. (when not ``sqlite``)
- ``ROUNDCUBE_DB_NAME``: the database name for roundcube service. (when not ``sqlite``)
.. _`roundcube documentation`: https://github.com/roundcube/roundcubemail/blob/master/config/defaults.inc.php#L28
Webmail settings
----------------

19
docs/database.rst
View File

@@ -1,8 +1,10 @@
Changing the database back-end
==============================
By default Mailu uses a SQLite database. We have changed the internals of Mailu
to enable the support of alternative database solutions such as PostgreSQL and MySQL/MariaDB.
By default Mailu uses a SQLite database. It is possible to use an alternative database solutions such as PostgreSQL and MySQL/MariaDB.
The Mailu database contains static data. SQLite is sufficient for any deployment scenario of Mailu. There is no need to use a different database system.
The Mailu development team recommends to use SQLite.
Migrating to a different database back-end
@@ -17,7 +19,7 @@ To switch to a different database back-end:
1. Run config-export to export the configuration. E.g. `docker compose exec admin flask mailu config-export --secrets --output mail-config.yml`
2. Set up your new database server. Refer to the subsequent sections for tips for creating the database.
3. Modify the database settings (DB_*) in mailu.env. Refer to the :ref:`configuration guide (link) <db_settings>` for the exact settings.
3. Modify the database settings (SQLAlchemy database URL) in mailu.env. Refer to the :ref:`configuration guide (link) <db_settings>` for the exact settings.
4. Start your Mailu deployment.
5. Run config-import to import the configuration. E.g. `docker exec -i $(docker compose ps -q admin) flask mailu config-import -v < mail-config.yml`
@@ -80,7 +82,7 @@ In ``pg_hba.conf`` there should be a line like this:
host mailu mailu <mailu_host>/32 md5
Note that this example is the bare-minimum to get Mailu working. Additional work needs to be
Note that this example is the bare-minimum to get Mailu working. Additional work needs to be
done by the database admin to setup their own means of backups and TLS encrypted connections.
Nowadays it is recommended to use the official PostgreSQL image from the PostgreSQL community. The repository is located `here <https://hub.docker.com/_/postgres>`_.
@@ -91,10 +93,10 @@ Mailu PostgreSQL
----------------
Mailu optionally came with a pre-configured PostgreSQL image which was deprecated in Mailu 1.8.
Since Mailu 1.9 it is removed from Mailu. The following section describes how to move to a different
Since Mailu 1.9 it is removed from Mailu. The following section describes how to move to a different
PostgreSQL image for novice administrators. The official PostgreSQL image (Postgres) will be used.
A Mailu deployment with the Mailu PostgreSQL image, will only use PostgreSQL for the Admin container
A Mailu deployment with the Mailu PostgreSQL image, will only use PostgreSQL for the Admin container
(Web administration interface). Roundcube uses SQLite as database back-end.
Mailu uses the following configuration for connecting to the database:
@@ -192,10 +194,7 @@ to
.. code-block:: docker
DB_HOST=database
DB_PORT=5432
DB_USER=mailu
DB_NAME=mailu
SQLALCHEMY_DATABASE_URI=postgresql://mailu:mailu@database/mailu
Mailu is now configured to use the official PostgreSQL docker image. Bring your new deployment online

18
setup/flavors/compose/mailu.env
View File

@@ -192,21 +192,3 @@ DEFAULT_SPAM_THRESHOLD=80
# This is a mandatory setting for using the RESTful API.
API_TOKEN={{ api_token }}
###################################
# Database settings
###################################
DB_FLAVOR={{ db_flavor }}
{% if db_flavor == 'postgresql' or db_flavor == 'mysql' %}
DB_USER={{ db_user }}
DB_PW={{ db_pw }}
DB_HOST={{ db_url }}
DB_NAME={{ db_name }}
{% endif %}
{% if ((db_flavor_rc == 'postgresql' or db_flavor_rc == 'mysql')) and webmail_type == 'roundcube' %}
ROUNDCUBE_DB_FLAVOR={{ db_flavor_rc }}
ROUNDCUBE_DB_USER={{ roundcube_db_user }}
ROUNDCUBE_DB_PW={{ roundcube_db_pw }}
ROUNDCUBE_DB_HOST={{ roundcube_db_url }}
ROUNDCUBE_DB_NAME={{ roundcube_db_name }}
{% endif %}

45
setup/templates/steps/database.html
View File

@@ -1,45 +0,0 @@
{% call macros.panel("info", "Database preferences") %}
<div class="form-group">
<label>Which database back end would you like to use?</label>
<br/>
<select class="btn btn-primary dropdown-toggle" name="db_flavor" id="database">
{% for dbflavor in ["sqlite", "postgresql", "mysql"] %}
<option value="{{ dbflavor }}" >{{ dbflavor }}</option>
{% endfor %}
</select>
<div class="form-group" id="external_db" style="display: none">
<p>Set external database parameters for <b>Admin UI</b></p>
<label>DB User</label>
<input class="form-control" type="text" name="db_user" placeholder="Username" id="db_user">
<label>Db Password</label>
<input class="form-control" type="password" name="db_pw" placeholder="Password" id="db_pw">
<label>Db URL (for using a different port use the notation host:port )</label>
<input class="form-control" type="text" name="db_url" placeholder="URL" id="db_url">
<label>Db Name</label>
<input class="form-control" type="text" name="db_name" placeholder="Database Name" id="db_name">
<br/>
<div class="form-group" id="db_flavor_rc_sel" style="display: none">
<label>Which database back end would you like to use for roundcube?</label>
<br/>
<select class="btn btn-primary dropdown-toggle" name=db_flavor_rc id="database_rc">
{% for dbflavor_rc in ["sqlite", "postgresql", "mysql"] %}
<option value="{{ dbflavor_rc }}" >{{ dbflavor_rc }}</option>
{% endfor %}
</select>
<div class="form-group" id="roundcube_external_db" style="display: none">
<p>Set external database parameters for <b>Roundcube</b></p>
<label>DB User</label>
<input class="form-control" type="text" name="roundcube_db_user" placeholder="Username" id="roundcube_db_user">
<label>DB Password</label>
<input class="form-control" type="password" name="roundcube_db_pw" placeholder="Password" id="roundcube_db_pw">
<label>DB URL (for using a different port use the notation host:port )</label>
<input class="form-control" type="text" name="roundcube_db_url" placeholder="URL" id="roundcube_db_url">
<label>DB Name</label>
<input class="form-control" type="text" name="roundcube_db_name" placeholder="Database Name" id="roundcube_db_name">
</div>
</div>
</div>
</div>
{% endcall %}

1
setup/templates/wizard.html
View File

@@ -14,7 +14,6 @@
{%for file in steps %}
{% include "steps/" + flavor + "/" + file %}
{% endfor %}
{% include "steps/database.html" %}
<input class="btn btn-primary" type="submit" value="Setup Mailu">
</form>
{% endblock %}

17
towncrier/newsfragments/2533.misc Normal file
View File

@@ -0,0 +1,17 @@
Introduce SQLAlchemy database uris for configuring the admin and roundcube database.
Remove the database configuration option from the setup utility. Using a different
database system than SQLite is not necessary for Mailu. The Mailu database generally
contains static data.
The usage of the *DB_* environment variables is deprecated now.
They can still be used in the release after Mailu 1.9, but will be removed
after that version. This means it will be removed from master after the upcoming
Mailu release.
To start using the new environment variables, all *DB_* environment variables must be changed to:
SQLALCHEMY_DATABASE_URI=<SQLAlchemy database URL>
SQLALCHEMY_DATABASE_URI_ROUNDCUBE=<Roundcube database URL>
If no URI is specified, SQLite is used with these settings:
SQLALCHEMY_DATABASE_URI=sqlite:////data/main.db
SQLALCHEMY_DATABASE_URI_ROUNDCUBE=sqlite:////data/roundcube.db

5
webmails/start.py
View File

@@ -26,7 +26,8 @@ with open("/etc/resolv.conf") as handle:
resolver = content[content.index("nameserver") + 1]
context["RESOLVER"] = f"[{resolver}]" if ":" in resolver else resolver
db_flavor = env.get("ROUNDCUBE_DB_FLAVOR", "sqlite")
db_uri = env.get("SQLALCHEMY_DATABASE_URI_ROUNDCUBE", "sqlite:////data/roundcube.db")
db_flavor = env.get("ROUNDCUBE_DB_FLAVOR")
if db_flavor == "sqlite":
context["DB_DSNW"] = "sqlite:////data/roundcube.db"
elif db_flavor == "mysql":
@@ -43,6 +44,8 @@ elif db_flavor == "postgresql":
env.get("ROUNDCUBE_DB_HOST", "database"),
env.get("ROUNDCUBE_DB_NAME", "roundcube")
)
elif db_uri:
context["DB_DSNW"] = db_uri
else:
print(f"Unknown ROUNDCUBE_DB_FLAVOR: {db_flavor}", file=sys.stderr)
exit(1)