With https://github.com/Mailu/Mailu/pull/3701 the remark was left to set the DB collation for MariaDB setups. However, the Helm chart has no option to overwrite the SQLAlchemy URI really. It selfs DB_USER, DB_PW, DB_NAME and DB_HOST and thus triggers the Alchemy URI to be overwritten (and it overwrites it statically as well...).
This commit adds the parameter / environment variable DB_APPENDIX allowing for, e.g., '?collation=utf8mb4_unicode_ci' to be set using an environment variable which the Helm chart can then set.
3722: Ensure we always use Mailu for sending emails in thunderbird r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Set useGlobalPreferredServer=false in autoconfig to ensure we always use Mailu's SMTP if there is more than one account configured.
The previous behaviour made no sense; it was set that way because the template at https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat makes it the default.
### Related issue(s)
- close#3721
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3709: Clarify ip listen address setting r=mergify[bot] a=nextgens
## What type of PR?
documentation
## What does this PR do?
Clarify ip listen address setting in setup to avoid open-relays.
Thanks to `@Cenness` for reporting it and suggesting a better wording.
### Related issue(s)
- closes#3680
- closes#3683
- #3690
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3440: Update reverse proxy documentation for using Traefik on a different host r=mergify[bot] a=Diman0
## What type of PR?
documentation
## What does this PR do?
It adds an extra section to the reverse proxy documentation. It provides an example on how to use Traefik on a different host than the host running Mailu. Now we will have documented both use cases where the reverse proxy is on the same host or a different host than Mailu.
### Related issue(s)
n/a
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
3699: Add the mariadb connector as per 3449 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Add the mariadb connector as per #3449.
MariaDB has no support for utf8mb4_0900_ai_ci which is the new default since MySQL version 8.0. In the current sqlalchemy version shipped with mailu, the mysqlconnector sets utf8mb4_0900_ai_ci as the collation to use when connecting. This causes all MariaDB connections to fail.
To fix the issue, either use the right connector or ensure it's configured with the right collation:
```
SQLALCHEMY_DATABASE_URI=mysql+mysqlconnector://<user>:<passwd>`@<host>/<database>?collation=utf8mb4_unicode_ci`
```
### Related issue(s)
- closes#3449
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3696: Include sensible error messages for LMTP protocol r=mergify[bot] a=fmos
Running into the rate limit yields difficult to debug log messages by the smtp container. Specifically the `Temporary user lookup failure` message by the smtp container is misleading.
## Example
Although this is running on Podman, the bugs are in the Python code and almost certainly are not influenced by the host infrastructure. (Leaving aside that I likely have a configuration problem, because the client IP address is not passed along correctly. But the present fix applies nevertheless and is not related to any specific cause of the rate limit triggering.)
### smtp logs
```shell
> podman logs --since "2024-12-25T07:33:31" --until "2024-12-25T07:33:33" systemd-mail-smtp
Dec 25 08:33:31 example postfix/smtpd[398]: connect from front[10.115.0.96]
INFO:root:Connect
Dec 25 08:33:31 example postfix/smtpd[398]: 6774324DE71C1: client=systemd-mail-front[10.115.0.96]
INFO:root:Connect
Dec 25 08:33:31 example postfix/cleanup[428]: 6774324DE71C1: message-id=<CAPhkJv+GTxVtwn6eNbBzPscohn6fgkhrYd2gEpUm2prr-5_7bg@mail.gmail.com>
Dec 25 08:33:32 example postfix/qmgr[376]: 6774324DE71C1: from=<SRS0=O1up=TS=gmail.com=fabiamos@example.com>, size=3968, nrcpt=1 (queue active)
Dec 25 08:33:32 example postfix/lmtp[429]: 6774324DE71C1: host front[10.115.0.96] said: 451 4.3.0 <fabian@example.com> Temporary user lookup failure (in reply to RCPT TO command)
Dec 25 08:33:32 example postfix/lmtp[429]: connect to front[10.115.0.9]:2525: Connection refused
Dec 25 08:33:32 example postfix/lmtp[429]: 6774324DE71C1: to=<fabian@example.com>, orig_to=<me+fancy@example.com>, relay=none, delay=0.63, delays=0.61/0.01/0.01/0, dsn=4.4.1, status=deferred (connect to front[10.115.0.9]:2525: Connection refused)
```
### admin logs
```shell
> podman logs --since "2024-12-25T07:33:31" --until "2024-12-25T07:33:33" systemd-mail-admin
10.115.0.96 - - [25/Dec/2024:08:33:31 +0100] "GET /internal/auth/email HTTP/1.0" 200 0 "-" "-"
[2024-12-25 08:33:32,030] WARNING in limiter: Authentication attempt from 10.115.0.99 has been rate-limited.
[2024-12-25 08:33:32,030] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
File "/app/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/mailu/internal/views/auth.py", line 27, in nginx_authentication
status, code = nginx.get_status(flask.request.headers['Auth-Protocol'], 'ratelimit')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/mailu/internal/nginx.py", line 140, in get_status
return status, codes[protocol]
~~~~~^^^^^^^^^^
KeyError: 'lmtp'
10.115.0.96 - - [25/Dec/2024:08:33:32 +0100] "GET /internal/auth/email HTTP/1.0" 200 0 "-" "-"
```
## What type of PR?
bug-fix
## What does this PR do?
### Related issue(s)
None
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly - not an enhancement
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file. - is a minor change
Co-authored-by: Fabian Stanke <me+github@fmos.at>
3691: Ensure mobileconfig has the right content-type r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure Apple mobileconfig is served using the right Content-Type
### Related issue(s)
- #3684
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3678: alpine3.21.0 r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Upgrade to alpine-3.21.0
### Related issue(s)
- #3677
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3650: Don't check empty passwords against HIBP r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Don't check empty passwords against HIBP; Apparently some password managers will trigger a race condition otherwise
### Related issue(s)
- closes#3633
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3647: upgrade to snuffleupagus v0.11 r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
- upgrade snuffleupagus to v0.11
- filter php:// wrapper types and only allow those we currently require (hardening)
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3648: Upgrade snappymail to v2.38.2 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Upgrade snappymail to v2.38.2. This is a security fix for [GHSA-2rq7-79vp-ffxm](https://github.com/the-djmaze/snappymail/security/advisories/GHSA-2rq7-79vp-ffxm) (mXSS)
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3622: alpine 3.20.3 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Upgrade from alpine 3.20.0 to alpine 3.20.3 ; We need a fix for [CVE-2024-5535](https://security.alpinelinux.org/vuln/CVE-2024-5535)
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3618: Filter logs line based and in binary mode without decoding utf-8 r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
try at fixing decoding errors when filtering logs
### Related issue(s)
- closes#3398
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
3608: Ensure healthchecks timeout r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure healthchecks timeout
### Related issue(s)
- close#3398
- close#3602
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3613: Fix http2 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Fix http2; it does not require IPv6 nor certificates
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3558: Only enable HARDENED_MALLOC with kernel 6.1+ r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Disable HARDENED_MALLOC unless the requirements are met
### Related issue(s)
- #3398
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3557: Fix#3531 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure we have both RSA and ECDSA certs when using letsencrypt now that the default behaviour from certbot has changed.
This is only important for new installs, not those renewing existing certs.
### Related issue(s)
- closes#3531
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3465: Maybe fix#3402 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Fix a potential problem with SO_REUSEADDR that may prevent admin from starting up
### Related issue(s)
- close#3402
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3468: Need sleep. r=nextgens a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Fix broken overrides introduced in 2024.06.17
### Related issue(s)
- closes#3467
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3466: Improve as per @ghostwheel42's suggestion r=mergify[bot] a=nextgens
Improve #3425 as per review
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
