3696: Include sensible error messages for LMTP protocol r=mergify[bot] a=fmos
Running into the rate limit yields difficult to debug log messages by the smtp container. Specifically the `Temporary user lookup failure` message by the smtp container is misleading.
## Example
Although this is running on Podman, the bugs are in the Python code and almost certainly are not influenced by the host infrastructure. (Leaving aside that I likely have a configuration problem, because the client IP address is not passed along correctly. But the present fix applies nevertheless and is not related to any specific cause of the rate limit triggering.)
### smtp logs
```shell
> podman logs --since "2024-12-25T07:33:31" --until "2024-12-25T07:33:33" systemd-mail-smtp
Dec 25 08:33:31 example postfix/smtpd[398]: connect from front[10.115.0.96]
INFO:root:Connect
Dec 25 08:33:31 example postfix/smtpd[398]: 6774324DE71C1: client=systemd-mail-front[10.115.0.96]
INFO:root:Connect
Dec 25 08:33:31 example postfix/cleanup[428]: 6774324DE71C1: message-id=<CAPhkJv+GTxVtwn6eNbBzPscohn6fgkhrYd2gEpUm2prr-5_7bg@mail.gmail.com>
Dec 25 08:33:32 example postfix/qmgr[376]: 6774324DE71C1: from=<SRS0=O1up=TS=gmail.com=fabiamos@example.com>, size=3968, nrcpt=1 (queue active)
Dec 25 08:33:32 example postfix/lmtp[429]: 6774324DE71C1: host front[10.115.0.96] said: 451 4.3.0 <fabian@example.com> Temporary user lookup failure (in reply to RCPT TO command)
Dec 25 08:33:32 example postfix/lmtp[429]: connect to front[10.115.0.9]:2525: Connection refused
Dec 25 08:33:32 example postfix/lmtp[429]: 6774324DE71C1: to=<fabian@example.com>, orig_to=<me+fancy@example.com>, relay=none, delay=0.63, delays=0.61/0.01/0.01/0, dsn=4.4.1, status=deferred (connect to front[10.115.0.9]:2525: Connection refused)
```
### admin logs
```shell
> podman logs --since "2024-12-25T07:33:31" --until "2024-12-25T07:33:33" systemd-mail-admin
10.115.0.96 - - [25/Dec/2024:08:33:31 +0100] "GET /internal/auth/email HTTP/1.0" 200 0 "-" "-"
[2024-12-25 08:33:32,030] WARNING in limiter: Authentication attempt from 10.115.0.99 has been rate-limited.
[2024-12-25 08:33:32,030] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
File "/app/venv/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/venv/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/app/venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/mailu/internal/views/auth.py", line 27, in nginx_authentication
status, code = nginx.get_status(flask.request.headers['Auth-Protocol'], 'ratelimit')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/mailu/internal/nginx.py", line 140, in get_status
return status, codes[protocol]
~~~~~^^^^^^^^^^
KeyError: 'lmtp'
10.115.0.96 - - [25/Dec/2024:08:33:32 +0100] "GET /internal/auth/email HTTP/1.0" 200 0 "-" "-"
```
## What type of PR?
bug-fix
## What does this PR do?
### Related issue(s)
None
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly - not an enhancement
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file. - is a minor change
Co-authored-by: Fabian Stanke <me+github@fmos.at>
3691: Ensure mobileconfig has the right content-type r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure Apple mobileconfig is served using the right Content-Type
### Related issue(s)
- #3684
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3678: alpine3.21.0 r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Upgrade to alpine-3.21.0
### Related issue(s)
- #3677
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3650: Don't check empty passwords against HIBP r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Don't check empty passwords against HIBP; Apparently some password managers will trigger a race condition otherwise
### Related issue(s)
- closes#3633
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3647: upgrade to snuffleupagus v0.11 r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
- upgrade snuffleupagus to v0.11
- filter php:// wrapper types and only allow those we currently require (hardening)
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3648: Upgrade snappymail to v2.38.2 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Upgrade snappymail to v2.38.2. This is a security fix for [GHSA-2rq7-79vp-ffxm](https://github.com/the-djmaze/snappymail/security/advisories/GHSA-2rq7-79vp-ffxm) (mXSS)
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3622: alpine 3.20.3 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Upgrade from alpine 3.20.0 to alpine 3.20.3 ; We need a fix for [CVE-2024-5535](https://security.alpinelinux.org/vuln/CVE-2024-5535)
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3618: Filter logs line based and in binary mode without decoding utf-8 r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
try at fixing decoding errors when filtering logs
### Related issue(s)
- closes#3398
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
3608: Ensure healthchecks timeout r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure healthchecks timeout
### Related issue(s)
- close#3398
- close#3602
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3613: Fix http2 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Fix http2; it does not require IPv6 nor certificates
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3558: Only enable HARDENED_MALLOC with kernel 6.1+ r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Disable HARDENED_MALLOC unless the requirements are met
### Related issue(s)
- #3398
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3557: Fix#3531 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure we have both RSA and ECDSA certs when using letsencrypt now that the default behaviour from certbot has changed.
This is only important for new installs, not those renewing existing certs.
### Related issue(s)
- closes#3531
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3465: Maybe fix#3402 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Fix a potential problem with SO_REUSEADDR that may prevent admin from starting up
### Related issue(s)
- close#3402
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3468: Need sleep. r=nextgens a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Fix broken overrides introduced in 2024.06.17
### Related issue(s)
- closes#3467
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3466: Improve as per @ghostwheel42's suggestion r=mergify[bot] a=nextgens
Improve #3425 as per review
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3463: Fix#3450 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure we can do more than 100 parallel sessions.
This will use more RAM.
### Related issue(s)
- close#3450
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
3441: Update Czech translation r=mergify[bot] a=kunago
## What type of PR?
Czech translation update
## What does this PR do?
### Related issue(s)
- None
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: kunago <miris@kunago.com>
3434: Fix#3425 r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Warn if passwords set from the UI contain leading or trailing whitespaces.
Not sure if it's worth backporting it.
### Related issue(s)
- close#3425
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3423: Fix#3411 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure we don't nuke all web-sessions when a password is changed.
This was reported with the command line but I suspect the API is affected too.
### Related issue(s)
- close#3411
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
