2021-03-01 23:41:19 +13:00
---
2021-03-11 20:41:24 +01:00
title: 'Advanced | Email Gathering with Fetchmail'
2021-03-01 23:41:19 +13:00
---
2024-05-02 19:44:54 +12:00
To enable the [fetchmail][fetchmail-website] service to retrieve e-mails, set the environment variable `ENABLE_FETCHMAIL` to `1` . Your `compose.yaml` file should look like following snippet:
2016-08-21 14:02:06 +02:00
2017-10-05 08:30:23 +02:00
```yaml
2016-08-21 14:03:37 +02:00
environment:
- ENABLE_FETCHMAIL=1
2017-10-05 08:30:23 +02:00
- FETCHMAIL_POLL=300
2016-08-21 14:02:06 +02:00
```
2023-04-10 15:36:34 +02:00
Generate a file called `fetchmail.cf` and place it in the `docker-data/dms/config/` folder. Your DMS folder should look like this example:
2016-08-22 10:16:47 +02:00
2021-03-01 23:41:19 +13:00
```txt
docs(chore): Normalize for consistency (#2206)
"Brief" summary/overview of changes. See the PR discussion or individual commits from the PR for more details.
---
Only applies to the `docs/content/**` content (_and `setup` command_). `target/` and `test/` can be normalized at a later date.
* Normalize to `example.com`
- Domains normalized to `example.com`: `mywebserver.com`, `myserver.tld`, `domain.com`, `domain.tld`, `mydomain.net`, `my-domain.tld`, `my-domain.com`, `example.org`, `whoami.com`.
- Alternative domains normalized to `not-example.com`: `otherdomain.com`, `otherdomain.tld`, `domain2.tld`, `mybackupmx.com`, `whoareyou.org`.
- Email addresses normalized to `admin@example.com` (in `ssl.md`): `foo@bar.com`, `yourcurrentemail@gmail.com`, `email@email.com`, `admin@domain.tld`.
- Email addresses normalized to `external-account@gmail.com`: `bill@gates321boom.com`, `external@gmail.com`, `myemail@gmail.com`, `real-email-address@external-domain.com`.
- **`faq.md`:** A FAQ entry title with `sample.domain.com` changed to `subdomain.example.com`.
- **`mail-fetchmail.md`:** Config examples with FQDNs for `imap`/`pop3` used `example.com` domain for a third-party, changed to `gmail.com` as more familiar third-party/external MTA.
* Normalize config volume path
- Normalizing local config path references to `./docker-data/dms/config/`: `./config/`, `config/`, \``config`\`, `/etc/` (_volume mount src path prefix_).
- Normalize DMS volume paths to `docker-data/dms/mail-{data,state,log}`: `./mail`, `./mail-state` `./data/mail`, `./data/state`, `./data/logs`, `./data/maildata`, `./data/mailstate`, `./data/maillogs`, (_dropped/converted data volumes: `maildata`, `mailstate`_).
- Other docker images also adopt the `docker-data/{service name}/` prefix.
* `ssl.md` - Use `dms/custom-certs` where appropriate.
* Apply normalizations to README and example `docker-compose.yml`
---
Common terms, sometimes interchangeably used or now invalid depending on context: `mail`, `mail container`, `mail server`, `mail-server`, `mailserver`,`docker-mailserver`, `Docker Mailserver`.
Rough transformations applied to most matches (_conditionally, depending on context_):
- 'Docker Mailserver' => '`docker-mailserver`'
- 'mail container' => '`docker-mailserver`' (_optionally retaining ' container'_)
- 'mail server' => 'mail-server' / '`docker-mailserver`'
- 'mail-server' => '`docker-mailserver`'
- 'mailserver' => 'mail-server' / '`docker-mailserver`'
Additionally I checked `docker run` (_plus `exec`, `logs`, etc, sub-commands_) and `docker-compose` commands. Often finding usage of `mail` instead of the expected `mailserver`
Additionally changes `mailserver` hostname in k8s to `mail` to align with other non-k8s examples.
---
* drive-by revisions
Mostly minor revisions or improvements to docs that aren't related to normalization effort.
2021-09-23 11:29:37 +12:00
├── docker-data/dms/config
2016-08-22 10:16:47 +02:00
│ ├── dovecot.cf
│ ├── fetchmail.cf
│ ├── postfix-accounts.cf
│ └── postfix-virtual.cf
2024-05-02 19:44:54 +12:00
└── compose.yaml
2016-08-22 10:16:47 +02:00
```
2016-08-21 14:10:33 +02:00
2021-03-01 23:41:19 +13:00
## Configuration
2016-08-22 10:23:51 +02:00
2024-05-02 19:44:54 +12:00
Configuration options for `fetchmail.cf` are covered at the [official fetchmail docs][fetchmail-docs-config] (_see the section "The run control file" and the table with "keyword" column for all settings_).
2016-08-22 10:40:34 +02:00
2024-05-02 19:44:54 +12:00
!!! example "Basic `fetchmail.cf` configuration"
2016-08-21 14:10:33 +02:00
2024-05-02 19:44:54 +12:00
Retrieve mail from `remote-user@somewhere.com` and deliver it to `dms-user@example.com` :
2016-08-21 14:10:33 +02:00
2021-03-02 17:39:06 +01:00
```fetchmailrc
2024-05-02 19:44:54 +12:00
poll 'mail.somewhere.com'
proto imap
user 'remote-user'
pass 'secret'
is 'dms-user@example .com'
2021-03-02 17:39:06 +01:00
```
2024-05-02 19:44:54 +12:00
- `poll` sets the remote mail server to connect to retrieve mail from.
- `proto` lets you connect via IMAP or POP3.
- `user` and `pass` provide the login credentials for the remote mail service account to access.
- `is` configures where the fetched mail will be sent to (_eg: your local DMS account in `docker-data/dms/config/postfix-accounts.cf` _).
2021-03-02 17:39:06 +01:00
2024-05-02 19:44:54 +12:00
---
2021-03-02 17:39:06 +01:00
2024-05-02 19:44:54 +12:00
??? warning "`proto imap` will still delete remote mail once fetched"
2016-08-21 14:10:33 +02:00
2024-05-02 19:44:54 +12:00
This is due to a separate default setting `no keep` . Adding the setting `keep` to your config on a new line will prevent deleting the remote copy.
docs(chore): Normalize for consistency (#2206)
"Brief" summary/overview of changes. See the PR discussion or individual commits from the PR for more details.
---
Only applies to the `docs/content/**` content (_and `setup` command_). `target/` and `test/` can be normalized at a later date.
* Normalize to `example.com`
- Domains normalized to `example.com`: `mywebserver.com`, `myserver.tld`, `domain.com`, `domain.tld`, `mydomain.net`, `my-domain.tld`, `my-domain.com`, `example.org`, `whoami.com`.
- Alternative domains normalized to `not-example.com`: `otherdomain.com`, `otherdomain.tld`, `domain2.tld`, `mybackupmx.com`, `whoareyou.org`.
- Email addresses normalized to `admin@example.com` (in `ssl.md`): `foo@bar.com`, `yourcurrentemail@gmail.com`, `email@email.com`, `admin@domain.tld`.
- Email addresses normalized to `external-account@gmail.com`: `bill@gates321boom.com`, `external@gmail.com`, `myemail@gmail.com`, `real-email-address@external-domain.com`.
- **`faq.md`:** A FAQ entry title with `sample.domain.com` changed to `subdomain.example.com`.
- **`mail-fetchmail.md`:** Config examples with FQDNs for `imap`/`pop3` used `example.com` domain for a third-party, changed to `gmail.com` as more familiar third-party/external MTA.
* Normalize config volume path
- Normalizing local config path references to `./docker-data/dms/config/`: `./config/`, `config/`, \``config`\`, `/etc/` (_volume mount src path prefix_).
- Normalize DMS volume paths to `docker-data/dms/mail-{data,state,log}`: `./mail`, `./mail-state` `./data/mail`, `./data/state`, `./data/logs`, `./data/maildata`, `./data/mailstate`, `./data/maillogs`, (_dropped/converted data volumes: `maildata`, `mailstate`_).
- Other docker images also adopt the `docker-data/{service name}/` prefix.
* `ssl.md` - Use `dms/custom-certs` where appropriate.
* Apply normalizations to README and example `docker-compose.yml`
---
Common terms, sometimes interchangeably used or now invalid depending on context: `mail`, `mail container`, `mail server`, `mail-server`, `mailserver`,`docker-mailserver`, `Docker Mailserver`.
Rough transformations applied to most matches (_conditionally, depending on context_):
- 'Docker Mailserver' => '`docker-mailserver`'
- 'mail container' => '`docker-mailserver`' (_optionally retaining ' container'_)
- 'mail server' => 'mail-server' / '`docker-mailserver`'
- 'mail-server' => '`docker-mailserver`'
- 'mailserver' => 'mail-server' / '`docker-mailserver`'
Additionally I checked `docker run` (_plus `exec`, `logs`, etc, sub-commands_) and `docker-compose` commands. Often finding usage of `mail` instead of the expected `mailserver`
Additionally changes `mailserver` hostname in k8s to `mail` to align with other non-k8s examples.
---
* drive-by revisions
Mostly minor revisions or improvements to docs that aren't related to normalization effort.
2021-09-23 11:29:37 +12:00
2024-05-02 19:44:54 +12:00
??? example "Multiple users or remote servers"
2016-08-22 22:30:47 +02:00
2024-05-02 19:44:54 +12:00
The official docs [config examples][fetchmail-config-examples] show a common convention to indent settings on subsequent lines for visually grouping per server.
2016-08-29 22:13:22 +02:00
2024-05-02 19:44:54 +12:00
=== "Minimal syntax"
2016-08-29 22:13:22 +02:00
2024-05-02 19:44:54 +12:00
```fetchmailrc
poll 'mail.somewhere.com' proto imap
user 'john.doe' pass 'secret' is 'johnny@example .com'
user 'jane.doe' pass 'secret' is 'jane@example .com'
2016-08-29 22:13:22 +02:00
2024-05-02 19:44:54 +12:00
poll 'mail.somewhere-else.com' proto pop3
user 'john.doe@somewhere -else.com' pass 'secret' is 'johnny@example .com'
```
=== "With optional syntax"
- `#` for adding comments.
- The config file may include "noise" keywords to improve readability.
```fetchmailrc
# Retrieve mail for users `john.doe` and `jane.doe` via IMAP at this remote mail server:
poll 'mail.somewhere.com' with proto imap wants:
user 'john.doe' with pass 'secret', is 'johnny@example .com' here
user 'jane.doe' with pass 'secret', is 'jane@example .com' here
2021-03-01 23:41:19 +13:00
2024-05-02 19:44:54 +12:00
# Also retrieve mail from this mail server (but via POP3).
# NOTE: This could also be all on a single line, or with each key + value as a separate line.
# Notice how the remote username includes a full email address,
# Some mail servers like DMS use the full email address as the username:
poll 'mail.somewhere-else.com' with proto pop3 wants:
user 'john.doe@somewhere -else.com' with pass 'secret', is 'johnny@example .com' here
```
!!! tip "`FETCHMAIL_POLL` ENV: Override default polling interval"
By default the fetchmail service will check every 5 minutes for new mail at the configured mail accounts.
```yaml
environment:
# The fetchmail polling interval in seconds:
FETCHMAIL_POLL: 60
```
2016-08-22 10:25:14 +02:00
2021-03-01 23:41:19 +13:00
## Debugging
2016-08-22 10:23:51 +02:00
2024-05-02 19:44:54 +12:00
To debug your `fetchmail.cf` configuration run this `setup debug` command:
2016-08-22 10:12:04 +02:00
2021-03-01 23:41:19 +13:00
```sh
2024-05-02 19:44:54 +12:00
docker exec -it dms-container-name setup debug fetchmail
2016-08-22 10:12:04 +02:00
```
2024-05-02 19:44:54 +12:00
??? example "Sample output of `setup debug fetchmail` "
```log
fetchmail: 6.3.26 querying outlook.office365.com (protocol POP3) at Mon Aug 29 22:11:09 2016: poll started
Trying to connect to 132.245.48.18/995...connected.
fetchmail: Server certificate:
fetchmail: Issuer Organization: Microsoft Corporation
fetchmail: Issuer CommonName: Microsoft IT SSL SHA2
fetchmail: Subject CommonName: outlook.com
fetchmail: Subject Alternative Name: outlook.com
fetchmail: Subject Alternative Name: *.outlook.com
fetchmail: Subject Alternative Name: office365.com
fetchmail: Subject Alternative Name: *.office365.com
fetchmail: Subject Alternative Name: *.live.com
fetchmail: Subject Alternative Name: *.internal.outlook.com
fetchmail: Subject Alternative Name: *.outlook.office365.com
fetchmail: Subject Alternative Name: outlook.office.com
fetchmail: Subject Alternative Name: attachment.outlook.office.net
fetchmail: Subject Alternative Name: attachment.outlook.officeppe.net
fetchmail: Subject Alternative Name: *.office.com
fetchmail: outlook.office365.com key fingerprint: 3A:A4:58:42:56:CD:BD:11:19:5B:CF:1E:85:16:8E:4D
fetchmail: POP3< +OK The Microsoft Exchange POP3 service is ready. [SABFADEAUABSADAAMQBDAEEAMAAwADAANwAuAGUAdQByAHAAcgBkADAAMQAuAHAAcgBvAGQALgBlAHgAYwBoAGEAbgBnAGUAbABhAGIAcwAuAGMAbwBtAA==]
fetchmail: POP3> CAPA
fetchmail: POP3< +OK
fetchmail: POP3< TOP
fetchmail: POP3< UIDL
fetchmail: POP3< SASL PLAIN
fetchmail: POP3< USER
fetchmail: POP3< .
fetchmail: POP3> USER user1@outlook .com
fetchmail: POP3< +OK
fetchmail: POP3> PASS *
fetchmail: POP3< +OK User successfully logged on.
fetchmail: POP3> STAT
fetchmail: POP3< +OK 0 0
fetchmail: No mail for user1@outlook .com at outlook.office365.com
fetchmail: POP3> QUIT
fetchmail: POP3< +OK Microsoft Exchange Server 2016 POP3 server signing off.
fetchmail: 6.3.26 querying outlook.office365.com (protocol POP3) at Mon Aug 29 22:11:11 2016: poll completed
fetchmail: normal termination, status 1
```
!!! tip "Troubleshoot with this reference `compose.yaml` "
[A minimal `compose.yaml` example][fetchmail-compose-example] demonstrates how to run two instances of DMS locally, with one instance configured with `fetchmail.cf` and the other to simulate a remote mail server to fetch from.
2021-03-01 23:41:19 +13:00
[fetchmail-website]: https://www.fetchmail.info
2024-05-02 19:44:54 +12:00
[fetchmail-docs-config]: https://www.fetchmail.info/fetchmail-man.html#the -run-control-file
[fetchmail-config-examples]: https://www.fetchmail.info/fetchmail-man.html#configuration -examples
[fetchmail-compose-example]: https://github.com/orgs/docker-mailserver/discussions/3994#discussioncomment -9290570