tests: Revise OAuth2 tests (#3795)

* tests: OAuth2 - Replace Python `/userinfo` endpoint with Caddy

Better documented, easier flow and separation of concerns via Caddy.

The python code had additional noise related to setting up a basic API which is abstracted away via `Caddyfile` config that's dedicated to this task.

* tests: OAuth2 - Minimize noise + Improve test assertion

Caddyfile can use an Access Token instead of a JWT. Much smaller and correct for this OAuth2 configuration. This new value has been documented inline.

Likewise the `sub` field returned is not important to this test. `email_verified` is kept as it may be helpful for further coverage testing.

The actual test-case has better assertions for success and failure by checking for Dovecot logs we expect instead of netcat response.

`oauth2` to `auth` for the Caddy container hostname is not necessary, just a more generic subdomain choice.

* tests: OAuth2 - Caddyfile `imap/xoauth2` route dynamic via query string

This way is more flexible and doesn't require modifying the `Caddyfile` directly, while still easy to use.

Additionally simplifies understanding the Caddyfile to maintainers by removing the `route` directive that was required to ensure a deterministic order of vars.

* tests: OAuth2 - `/imap/xoauth2` respond with IMAP commands for netcat

Since this is the only intended usage, might as well have it respond with the full file content.

* tests: OAuth2 - Implement coverage for `OAUTHBEARER`

Caddyfile route for `/imap/` now accepts any subpath to support handling both `xoauth2` and `oauthbearer` subpaths.

Both SASL mechanisms represent the same information, with `XOAUTH2` being a common mechanism to encounter defined by Google, whilst `OAUTHBEARER` is the newer variant standardized by RFC 7628 but not yet as widely adopted.

The request to `/userinfo` endpoint will be the same, only the `credentials` value to be encoded differs.

Instead of repeating the block for a similar route, this difference is handled via the Caddyfile `map` directive.

We match the path context (_`/xoauth2` or `/oauthbearer`, the `/imap` prefix was stripped by `handle_path` earlier_), when there is a valid match, `sasl_mechanism` and `credentials` map vars are created and assigned to be referenced by the later `respond` directive.

---

Repeat the same test-case logic, DRY with log asserts extracted to a common function call. This should be fine as the auth method will be sufficient to match against or a common failure caught.

* tests: OAuth2 - Minor revisions

Separate test cases and additional comment on creating the same base64 encoded credentials via CLI as an alternative to running Caddy.

Added a simple `compose.yaml` for troubleshooting or running the container for the `/imap/xoauth2` / `/imap/oauthbearer` endpoints.

* tests: OAuth2 - Route endpoints in Caddyfile with snippets instead

`reverse_proxy` was a bit more convenient, but the additional internal ports weren't really relevant. It also added noise to logging when troubleshooting.

The `import` directive with Snippet blocks instead is a bit cleaner, but when used in a single file snippets must be defined prior to referencing them with the `import` directive.

---

`compose.yaml` inlines the examples, with slight modification to `localhost:80`, since the Caddyfile examples `auth.example.test` is more relevant to the tests which can use it, and not applicable to troubleshooting locally outside of tests.

* chore: Add entry to `CHANGELOG.md`

* chore: Additional context on access token

test/tests/serial/mail_with_oauth2.bats

@@ -9,7 +9,7 @@ function setup_file() {
   export DMS_TEST_NETWORK='test-network-oauth2'
   export DMS_DOMAIN='example.test'
   export FQDN_MAIL="mail.${DMS_DOMAIN}"
-  export FQDN_OAUTH2="oauth2.${DMS_DOMAIN}"
+  export FQDN_OAUTH2="auth.${DMS_DOMAIN}"
 
   # Link the test containers to separate network:
   # NOTE: If the network already exists, test will fail to start.
@@ -19,20 +19,19 @@ function setup_file() {
   docker run --rm -d --name "${CONTAINER2_NAME}" \
     --hostname "${FQDN_OAUTH2}" \
     --network "${DMS_TEST_NETWORK}" \
-    --volume "${REPOSITORY_ROOT}/test/config/oauth2/:/app/" \
-    docker.io/library/python:latest \
-    python /app/provider.py
+    --volume "${REPOSITORY_ROOT}/test/config/oauth2/Caddyfile:/etc/caddy/Caddyfile:ro" \
+    caddy:2.7
 
-  _run_until_success_or_timeout 20 sh -c "docker logs ${CONTAINER2_NAME} 2>&1 | grep 'Starting server'"
+  _run_until_success_or_timeout 20 bash -c "docker logs ${CONTAINER2_NAME} 2>&1 | grep 'serving initial configuration'"
 
   #
   # Setup DMS container
   #
 
-  # Add OAUTH2 configuration so that Dovecot can reach out to our mock provider (CONTAINER2)
+  # Add OAuth2 configuration so that Dovecot can query our mocked identity provider (CONTAINER2)
   local ENV_OAUTH2_CONFIG=(
     --env ENABLE_OAUTH2=1
-    --env OAUTH2_INTROSPECTION_URL=http://oauth2.example.test/userinfo/
+    --env OAUTH2_INTROSPECTION_URL=http://auth.example.test/userinfo
   )
 
   export CONTAINER_NAME=${CONTAINER1_NAME}
@@ -49,6 +48,9 @@ function setup_file() {
 
   # Set default implicit container fallback for helpers:
   export CONTAINER_NAME=${CONTAINER1_NAME}
+
+  # An initial connection needs to be made first, otherwise the auth attempts fail
+  _run_in_container_bash 'nc -vz 0.0.0.0 143'
 }
 
 function teardown_file() {
@@ -56,11 +58,21 @@ function teardown_file() {
   docker network rm "${DMS_TEST_NETWORK}"
 }
 
-
-@test "oauth2: imap connect and authentication works" {
-  # An initial connection needs to be made first, otherwise the auth attempt fails
-  _run_in_container_bash 'nc -vz 0.0.0.0 143'
-
-  _nc_wrapper 'auth/imap-oauth2-auth.txt' '-w 1 0.0.0.0 143'
-  assert_output --partial 'Examine completed'
+@test "should authenticate with XOAUTH2 over IMAP" {
+  _nc_wrapper 'auth/imap-oauth2-xoauth2.txt' '-w 1 0.0.0.0 143'
+  __verify_successful_login 'XOAUTH2'
+}
+
+@test "should authenticate with OAUTHBEARER over IMAP" {
+  _nc_wrapper 'auth/imap-oauth2-oauthbearer.txt' '-w 1 0.0.0.0 143'
+  __verify_successful_login 'OAUTHBEARER'
+}
+
+function __verify_successful_login() {
+  local AUTH_METHOD=${1}
+
+  # Inspect the relevant Dovecot logs to catch failure / success:
+  _run_in_container grep 'dovecot:' /var/log/mail.log
+  refute_output --partial 'oauth2 failed: Introspection failed'
+  assert_output --partial "dovecot: imap-login: Login: user=<user1@localhost.localdomain>, method=${AUTH_METHOD}"
 }