2529: Improve fetchmail r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Improve fetchmail:
- allow delivery via LMTP (faster, bypassing the filters)
- allow several folders to be retrieved
- run fetchmail as non-root
- tweak the compose file to ensure we have all the dependencies
### Related issue(s)
- closes#1231
- closes#2246
- closes#711
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
2483: Introduce FETCHMAIL_ENABLED r=mergify[bot] a=DjVinnii
## What type of PR?
Enhancement
## What does this PR do?
Add `FETCHMAIL_ENABLED` to enable/disable the Fetchmail functionality in the Admin UI.
### Related issue(s)
- closes#2127
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
2535: fix the linux/arm/v7 build r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
The arm builder is running aarch64 ... and there is no package for arm/v7
Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2530: disable SESSION_COOKIE_SECURE when TLS_FLAVOR=notls r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
People are unlikely to proxy everything
### Related issue(s)
- closes#2527
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2514: Update deps r=mergify[bot] a=ghostwheel42
## What type of PR?
update python dependencies
## What does this PR do?
Update python deps in base image
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2479: Rework the anti-spoofing rule r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
We shouldn't assume that Mailu is the only MTA allowed to send emails on behalf of the domains it hosts.
We should also ensure that it's non-trivial for email-spoofing of hosted domains to happen
Previously we were preventing any spoofing of the envelope from; Now we are preventing spoofing of both the envelope from and the header from unless some form of authentication passes (is a RELAYHOST, SPF, DKIM, ARC)
### Related issue(s)
- close#2475
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2498: Implement ITERATE in podop r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
This makes ``doveadm -A`` work.
The easiest way to try it out is:
```
doveadm dict iter proxy:/tmp/podop.socket:auth shared/userdb
or
doveadm user '*'
```
The protocol is described at https://doc.dovecot.org/developer_manual/design/dict_protocol/
The current version of dovecot is not using flags... so there's little gain in implementing them.
### Related issue(s)
- close#2499
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2328: Feature: Configurable default spam threshold used for new users r=mergify[bot] a=enginefeeder101
## What type of PR?
Feature
## What does this PR do?
This PR adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` is
used for this purpose. When not set, it defaults back to 80%, as the
default value was before.
If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in. Should that be done?
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: enginefeeder101 <enginefeeder101@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2458: Fix: Don't update updated_at on quota_bytes_used change r=mergify[bot] a=DjVinnii
## What type of PR?
bug-fix
## What does this PR do?
This PR makes sure that the `updated_at` field is not updated when `quota_bytes_used` is updated. All other updates to the `User` model still updates the `updated_at` field.
This is done by explicitly using an method in the `Base` class triggering [`flag_modified`][url-flag-modified].
### Related issue(s)
- closes#1363
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
<!-- LINKS-->
[url-flag-modified]: https://docs.sqlalchemy.org/en/14/orm/session_api.html#sqlalchemy.orm.attributes.flag_modified
Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
2455: Fix/missing tanslations r=mergify[bot] a=DjVinnii
## What type of PR?
Fix/Enhancement
## What does this PR do?
Add missing Dutch translation, as well as the German translation for `Start of vacation`
### Related issue(s)
- closes#2217
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
This new advanced setting to harden cipher configuration on port 25. Changing the default is strongly discouraged, please read the documentation before doing so.
Antispam.rst contained a syntax error.
Move config description to common section which is more fitting.
Fixed wrong assignment of default value for DEFAULT_SPAM_THRESHOLD in models.py.
2404: Forwarding emails option in user settings did not support 1 letter do… r=mergify[bot] a=Diman0
…mains.
## What type of PR?
Bug-fix
## What does this PR do?
Forwarding emails option in user setting did not support 1 letter domains. The regex for checking the validity of multiple email addresses string has been modified to allow 1 letter domains and to allow 1 letter local part.
### Related issue(s)
- closes#2402
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [n/a] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Per requested changes added the ``DEFAULT_SPAM_THRESHOLD`` to the main
application configuration dictionary in ``configuration.py`` and updated
``models.py`` accordingly.
No error handling is added, as that was not required.
This commit adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` can
be used for this purpose. When not set, it defaults back to 80%, as the
default value was before
If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in.
2276: Autoconfig of email clients r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
It provides auto-configuration templates for email clients and encourages them to use implicit TLS (see https://nostarttls.secvuln.info/)
There are numerous caveats:
- it will only work if suitable DNS records are created and certificates obtained (autoconfig, autodiscover, ...)
- the mobileconfig file isn't signed
- the credentials will be prompted... we could/should provision a token on each request instead
- it currently doesn't advertise caldav
- it's IMAP only
### Related issue(s)
- close#224
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2210: Add input validation for domain creation r=mergify[bot] a=0pc0deFR
## What type of PR?
bug-fix
## What does this PR do?
This patch add the input validation for domain creation.
### Related issue(s)
- Mention an issue like: #1817
- Auto close an issue like: closes#1817
Co-authored-by: Kevin Falcoz <0pc0defr@gmail.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2254: Send ISRG_X1 on port 25, make DANE pin that r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure we send ISRG_X1 in the handshake on port 25 (non-interactive, size doesn't really matter).
Update the DANE pin to reflect the change.
I am not sure whether we will need to add --preferred-chain= in the future; This may be the case when letsencrypt decides to use X2/the ECDSA chain
This needs to be tested on a letsencrypt account that isn't mine (I'm opted in for the alternate cert chains)
### Related issue(s)
- closes#2138
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
There's already a towncrier news for it
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2116: fix 2114: redirect old path r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Old paths may still be cached in browsers, it's easy enough to redirect them
### Related issue(s)
- close#2114
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2094: Sessions tweaks r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
- Make all sessions permanent, introduce SESSION_TIMEOUT and PERMANENT_SESSION_LIFETIME.
- Prevent the creation of a session before there is a login attempt
- Ensure that webmail tokens are in sync with sessions
### Related issue(s)
- close#2080
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2044: Vault/rspamd: don't return any key for relayed domains r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR
Don't return any key for relayed domains. We may want to revisit this (ARC signing)... but in the meantime it saves from a scary message in rspamd.
```signing failure: cannot request data from the vault url: /internal/rspamd/vault/v1/dkim/ ...```
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2042: Add MESSAGE_RATELIMIT_EXEMPTION r=mergify[bot] a=nextgens
## What type of PR?
Enhancement
## What does this PR do?
Add a new knob called ```MESSAGE_RATELIMIT_EXEMPTION```.
### Related issue(s)
- #1774
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
We may want to revisit this (ARC signing)... but in the meantime
it saves from a scary message in rspamd
signing failure: cannot request data from the vault url: /internal/rspamd/vault/v1/dkim/ ...
this is working fine, but introduces a sqlalchemy warning
when using config-import:
/app/mailu/schemas.py:822:
SAWarning: Identity map already had an identity for (...),
replacing it with newly flushed object.
Are there load operations occurring inside of an event handler
within the flush?
ConfigManager should not replace app.config - this is causing trouble
with some other flask modules (swagger).
Updated ConfigManager to only modify app.config and not replace it.
1987: Enhancement to the rate limits r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Turn the rate-limiters into something useful (that won't fire for no reason).
- fix rate-limiting on /webdav/
- it changes the rate-limiting behaviour from limiting a single IP address to a subnet of a reasonable size (/24 on v4 and /56 on v6 both are now configurable) : AUTH_RATELIMIT_IP / AUTH_RATELIMIT_IP_V4_MASK / AUTH_RATELIMIT_IP_V6_MASK
- It ensures we only use IP-based rate-limits for attempts on accounts that do not exist
- it creates a new rate limit preventing attackers from targetting a specific user account (separate from what's above) : AUTH_RATELIMIT_USER
- it introduces a rate limiting exemption mechanism whereby, upon authentication, users will see their source-ip address being exempt for a specific amount of time AUTH_RATELIMIT_EXEMPTION_LENGTH. A similar mechanism is available for web-based sessions (see below)
- It introduces in AUTH_RATELIMIT_EXEMPTION a comma separated list of network CIDRs that will be exempt from both types of rate limiting
- it implements device-tokens, as described on https://owasp.org/www-community/Slow_Down_Online_Guessing_Attacks_with_Device_Cookies to ensure that genuine users aren't locked-out by a malicious attacker abusing the rate-limit feature.
Things that could be improved include:
- the IP-based rate limiter flags attempts against "non-existing" accounts: it could go further and flag the number of unique non-existing accounts attempted (to prevent the case of a user making a typo in his MUA configuration)
- the IP address exemption mechanism doesn't pin the exemption to a specific username: any real user can trivially bypass the rate limits (and attempt to brute-force someone else's account)
### Related issue(s)
- close#1926
- close#1745
- close#1915
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Diman0 <diman@huisman.xyz>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
2018: show dmarc record for report domain in domain details r=mergify[bot] a=ghostwheel42
## What type of PR?
documentation
## What does this PR do?
show dmarc record for report domain in domain details
### Related issue(s)
closes#1382
## Prerequisites
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1986: Document how to setup client autoconfig r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Document how to setup autoconfig. This works with most open-source MUAs (thunderbird, evolution, ...)
We could go further than that by providing dynamic configuration (issue an auth token for each MUA request)... but it won't work unless a new DNS entry (and matching certificate) is created.
### Related issue(s)
- #224
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
2014: Update Chinese translation r=mergify[bot] a=qy117121
## What type of PR?
translation
## What does this PR do?
Update Chinese translation. Use `zh` instead of `zh_CN`.
### Related issue(s)
none
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: qy117121 <mixuan121@gmail.com>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2017: rspamd: get dkim keys via REST API instead of filesystem r=mergify[bot] a=ghostwheel42
## What type of PR?
enhancement
## What does this PR do?
rspamd now uses hashicorp's vault api v1 to get dkim keys and selectors for a domain.
this allows future enhancement (multiple keys) without reconfiguring and restarting rspamd.
it also makes mounting the /dkim volume into the rspamd container unnecessary.
### Related issue(s)
- improves and closes#2012
- allows to implement key rotation using multiple selectors (see #1700)
- allows to implement dkim for alternate domains (see #1519)
- fixes and closes#1345 (selector transmitted by admin container is used)
- closes#1179 (no keys on disk)
- allows to implement key rotation from the outside (ie. via a helper script talking to some dns provider's api) (see #547)
## Prerequisites
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1997: Prevent traceback when using non-email in login r=mergify[bot] a=ghostwheel42
There's a traceback when the username used to log via SMTPAUTH
in is not an email address:
=== before ===
```
[...] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
context = constructor(dialect, self, conn, *args)
File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
param.append(processors[key](compiled_params[key]))
File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
return process_param(value, dialect)
File "/app/mailu/models.py", line 60, in process_bind_param
localpart, domain_name = value.lower().rsplit('`@',` 1)
ValueError: not enough values to unpack (expected 2, got 1)
[...]
[parameters: [{'%(140657157923216 param)s': 'foobar'}]]
```
=== after ===
```
[...] WARNING in nginx: Invalid user 'foobar': (builtins.ValueError) invalid email address (no "`@")`
```
## What type of PR?
enhancement
## What does this PR do?
replace traceback (ERROR) with error message (WARNING)
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2007: allow sending emails as user+detail@domain.tld r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix or enhancement
## What does this PR do?
Allows sending emails with an added "+detail" in the local part.
### Related issue(s)
closes#1948
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: root <ghostwheel42@users.noreply.github.com>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
