* docs: Relocate account / auth pages into a common section
* docs: Update references to relocated pages
* docs: Add account management overview page
Updates remaining links to account sections on this page instead (_for `accounts`, `aliases`, `quotas`_).
This page will cover the features and defer to separate pages for more specific content where relevant.
* docs: Correct relocated pages titles and links
* docs: Accounts (Dovecot Master) - Minor revisions
* docs: Fix highlighting roundcube PHP snippet in OAuth2 page
* docs: Accounts (File) - Refactor
- Manual method not necessary to document.
- Condense `setup` example guidance.
- Quotas / Aliases content migrated to Overview when not specific about file provisioner.
Some of the content is this commit is not a complete revision.
* chore: Temporary commit
* docs(refactor): Sub-addressing section
Much better docs on the sub-addressing feature supported by Postfix and Dovecot, along with the guidance with usage in Sieve.
* docs: Revise accounts section
Add some context regarding DMS accounts and their distinction/overlap from the email address functionality, and it's relevant context for receiving/sending.
File provisioner, minor revisions to referencing associated config files and account management.
* docs: Minor adjustments
* docs: Refactor the quota section
Better documented with links and coverage over the workaround details we've implemented.
* docs: Revise the quota section
Minor revisions with phrasing, admonitions for structure and better explanation of the feature functionality/purpose.
* docs: Alias section refactor
Extensively covers known issues and technical details that have been discussed often enough.
The improvements should benefit both users and maintainers.
* docs: Refactor master accounts page
This rewrite should more clearly document the feature, along with a better example and additional links for reference.
* docs: OAuth2 revision
Minor update to this page:
- Links extracted to bottom of page as per convention.
- ENV file example converted to preferred `compose.yaml` ENV settings.
* docs: Sieve minor revisions
- Correct link to subaddressing section
- Make the config file example snippets intended filename less ambiguous.
- Minor rephrasng.
* docs: Revise accounts overview section
Revised the account section and added additional clarity for common confusion with relation to sender address and multi-domain support.
Top of the page now clarifies it's a technical reference and directs users to the related pages for configuration / caveats.
Technical Overview links to Dovecot docs were missing.
* docs: Another revision pass
File based provisioner docs:
- Sections indent with info admonitions.
- Accounts section expanded with config format and example.
- Quotas section expanded and shifted to bottom (alphabetical sort).
- Split into `setup` CLI and config reference groups.
Overview page:
- Sections indent with info admonitions.
- Revised content.
* docs(chore): Shift sub-addressing section
This is related to accounts and aliases, but not provisioners, thus extract out of the accounts parent section.
* docs: Document `postfix-accounts.cf` third column
This lacked documentation but was community contributed feature to allow further customization of a Dovecot Account.
It has caveats as DMS does not take these into consideration anywhere in scripts. Documenting officially for better awareness.
* docs: Revise and expand supplementary pages
Better outline the OAuth2 login process, the two supported login mechanisms and their docs/rfcs, along with documenting caveat with mail client compatibility.
Add a verification tip for the OAuth2 support, showing how `curl` can be used, along with caveat presently affecting the `curl` in DMS v14.
Additionally note the feature still isn't documented fully, providing the user with additional references for more information.
`ACCOUNT_PROVISIONER` ENV docs minimized. No `OIDC` provisioner plans, the OAuth2 docs page now mentions SCIM 2.0 API as the next step towards resolving that concern. The tip admonition was removed as it no longer provides value, instead we link to the Account Management overview page.
Dovecot Master Accounts docs page now lightly document the `setup` CLI and config format for the feature.
* docs: Fix broken anchor links
Some anchor links to different parts of our docs have gone stale. This branch also broke a few itself that I missed.
The build now only reports issues with anchor links to Content Tabs, which it must not be aware of during the build (_MKDocs Material specific feature?_)
* docs(lint): Fix indentation level
* chore: Add entry to `CHANGELOG.md` + corrections
* fix: Update `dovecot-fts-xapian` to `1.7.13`
Contains a fix to a regression introduced that broke indexing
---------
Co-authored-by: casperklein <casperklein@users.noreply.github.com>
* update `compile.sh` and Dovecot FTS Xapian to 1.7.12
- I updated from 1.5.5. Moreover, I adjusted the script to have what I
consider better style.
* update Dockerfile to use recent updates
* update CHANGELOG
Fixes an issue with the Getmail service, view PR thread for additional details.
- Log an error when the expected service state directory doesn't exist.
- The location `/var/lib/getmail/` doesn't seem like it should have been introduced. Drop it in favor of `/tmp/docker-mailserver/getmail`. It appears to be for storing remote mail that was retrieved if not configured to send to Dovecot like our docs advise. This location was never valid anyway (_as referenced issue covers_).
Custom parameters must be referenced to be retained when `postconf -n` is run. If those parameters are referenced by `postfix-master.cf` this needs to update `master.cf` before updating `main.cf`.
* move `policies_group.conf` to correct location
I originally assumed the file had to be placed into `scores.d`, but I
now know that `local.d` is actually correct.
* add configuration for composite symbols
See updates to #3690:
Additional Rspamd Symbols
Rspamd has so-called composite symbols that trigger when a condition
is met. Especially AUTH_NA and AUTH_NA_OR_FAIL will adjust the scores
of various lines in the table above. This needs to be taken into account.
* update CHANGELOG
* remove leftover statement on `/etc/os-release`
* update wording on the PR template
* add section about other services to Rspamd docs
* remove more outdated information from Rspamd docs
* moved links and minor rewording in Rspamd docs
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* chore: `relay.sh` helper - Reference user config paths via variables
* chore: Better document postfix helper `_vhost_collect_postfix_domains()`
The functionality is effectively the same for the two configs for the most part when it comes to parsing out a domain from the target value.
Virtual aliases is more flexible in value, which may not have a domain-part present (manual user edit).
* chore: `check-for-change.sh` - Support VHOST change visibility
- Moves the "handle changes" logic into it's own scoped function, out of the main change detection loop logic.
- This will be benefit a future commit change that will rely on `VHOST_UPDATED=1`.
* chore: `relay.sh` - Minor revisions to minimize diff noise
- Better phrasing of the current logic comments.
- Regex patterns assigned to variables (easier to grok intention)
- Bulk of the logic for generating `/etc/postfix/relayhost_map` wrapped into a separate function with Postfix config setting handled separately.
* refactor: `relay.sh` opt-out logic
- Split the two distinct features that configure `/etc/postfix/relayhost_map` into separate functions (_`MATCH_VALID` var no longer needed for legacy support_).
- Instead of extracting domains from `postfix-accounts.cf` + `postfix-virtual.cf`, this has already been handled at `/etc/postfix/vhost`, sourcing from there is far less complicated.
- Rename loop var `DOMAIN_PART`to `SENDER_DOMAIN` for better context of what it represents when appended to the config file.
- Revised maintenance notes + guidance towards a future refactor of this relayhost feature support.
* docs: `relay.sh` - Additional comment revisions
* feat: `DEFAULT_RELAY_HOST` can now also use relay credentials ENV
- Remove comment regarding `smtp_sasl_password_maps = static:${RELAY_USER}:${RELAY_PASSWORD}`, it could be used but `main.cf` presently has `644` permissions vs the `sasl_passwd` file permissions of `600`, less secure at preventing leaking of secrets (ignoring the ENV exposure itself).
- Move the `main.cf` settings specific to relayhost credentials support / security into to the relevant function scope instead. This also allows for the configuration to be applied by a change detection event without container restart requirement.
- Outer functions for setup and change detection to call have a clearer config dependency guard, as does the `_legacy_support()`.
- These changes now support `DEFAULT_RELAY_HOST` to leverage the relay credentials ENV as well.
- `DATABASE_RELAYHOSTS` is available in scope to the functions called here that reference it.
* docs: Revised ENV docs on relay host config
Better quality guidance on configuring relay hosts.
* chore: Add entry to `CHANGELOG.md`
* fix: `relay.sh` - `grep` regex compatibility with `+` requires `-E`
* chore: `postfix.sh` - `FIRST_FIELD` => More descriptive field name
* docs: Better document DMS volumes
* docs: Remove any mention of `ONE_DIR` ENV
* chore: Remove `ONE_DIR` ENV from scripts
Only `ONE_DIR=0` has any effect. As the actual feature is now dependent upon the `/var/mail-state` location existing.
It is advised not mounting anything there instead if wanting to avoid runtime state consolidation.
* docs: Adjust link ref convention
This is more search friendly / organized to find references to all DMS volumes.
* lint: Ensure final newline is present
VSCode by default excludes this if the last line rendered is removed (rendered as a separate blank line).
A separate setting can enforce adding the final newline upon save regardless.
* only declare Rspamd vars when not already declared
* update CHANGELOG
* Update CHANGELOG.md
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* chore: Detect missing final newline in configs read
These lines will be not be processed by `read`, emit a warning to raise awareness.
* fix: Ensure parsed config has final newline appended (when possible)
This functionality was handled in `accounts.sh` via a similar sed command (that the linked references also offer).
`printf` is better for this, no shellcheck comment required either.
We additionally don't attempt to modify files that are read-only.
* fix: Ensure parsed configs have CRLF to LF corrected (where possible)
Likewise, this runtime fix was only covering two config files. It now applies to all callers of this method.
* fix: Sanitize `postfix-master.cf` via helper
This feature should have been using the helper to avoid user error from their config updates accidentally introducing subtle breakage implicitly (due to CRLF or missing final newline).
* tests: Add test cases for new helpers
* tests: `rm` is redundant when using `BATS_TEST_TMPDIR`
This temporary directory is created and removed implicitly. Even after a test failure.
* chore: Remove old `postfix-virtual.cf` migration logic
This was introduced in 2018, there should be no one needing to rely on this anymore?
* tests: Remove comment on sed failure concern
* chore: Add entry to `CHANGELOG.md`
* Apply suggestions from code review
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
---------
Co-authored-by: Georg Lauterbach <44545919+georglauterbach@users.noreply.github.com>
* move log/filter functions into own file
* add ShellCheck global directives
* use new function for tracking logs
The new function, called `_send_email_with_mid`, aligns with suggestions
from @polarethene and is heavily simplified compared to its predecessor
`_send_email_and_get_id`. New helpers will be introduced to filter logs
according to the MID constructed in this function.
* new filters for searching logs with MID
* use new filters (and sending) functions
* add new helper for asserting non-existence of log message
* use new filters in tests
* Apply suggestions from code review
- `_mid` / `MID` => `_msgid` / `MSG_ID`
- Revised documentation / tooltip comments
* Apply suggestions from code review
* fix tests
* use more distinct names for MSG_ID headers
* update `_filter_service_log` to not use `-i -E`
Moreover, I added a function to print the whole mail log. Appropriate
comments were added to this function to indicate that one should only
use this function when necessary.
* adjust helpers to new helper filter
* follow-up of previous commit
* add CHANGELOG entry
* Apply suggestions from code review
* chore: Update OAuth2 to use new log helper
* Apply suggestions from code review
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* added explicit `_regexp` filters for logs
* Apply suggestions from code review
---------
Co-authored-by: Brennan Kinney <5098581+polarathene@users.noreply.github.com>
* fix: Dovecot PassDB should restrict allowed auth mechanisms
This prevents PassDBs incompatible with certain auth mechanisms from logging failures which accidentally triggers Fail2Ban.
Instead only allow the PassDB to be authenticated against when it's compatible with the auth mechanism used.
* tests: Use `curl` for OAuth2 login test-cases instead of netcat
`curl` provides this capability for both IMAP and SMTP authentication with a bearer token. It supports both `XOAUTH2` and `OAUTHBEARER` mechanisms, as these updated test-cases demonstrate.
* chore: Add entry to `CHANGELOG.md`
* tests: OAuth2 - Replace Python `/userinfo` endpoint with Caddy
Better documented, easier flow and separation of concerns via Caddy.
The python code had additional noise related to setting up a basic API which is abstracted away via `Caddyfile` config that's dedicated to this task.
* tests: OAuth2 - Minimize noise + Improve test assertion
Caddyfile can use an Access Token instead of a JWT. Much smaller and correct for this OAuth2 configuration. This new value has been documented inline.
Likewise the `sub` field returned is not important to this test. `email_verified` is kept as it may be helpful for further coverage testing.
The actual test-case has better assertions for success and failure by checking for Dovecot logs we expect instead of netcat response.
`oauth2` to `auth` for the Caddy container hostname is not necessary, just a more generic subdomain choice.
* tests: OAuth2 - Caddyfile `imap/xoauth2` route dynamic via query string
This way is more flexible and doesn't require modifying the `Caddyfile` directly, while still easy to use.
Additionally simplifies understanding the Caddyfile to maintainers by removing the `route` directive that was required to ensure a deterministic order of vars.
* tests: OAuth2 - `/imap/xoauth2` respond with IMAP commands for netcat
Since this is the only intended usage, might as well have it respond with the full file content.
* tests: OAuth2 - Implement coverage for `OAUTHBEARER`
Caddyfile route for `/imap/` now accepts any subpath to support handling both `xoauth2` and `oauthbearer` subpaths.
Both SASL mechanisms represent the same information, with `XOAUTH2` being a common mechanism to encounter defined by Google, whilst `OAUTHBEARER` is the newer variant standardized by RFC 7628 but not yet as widely adopted.
The request to `/userinfo` endpoint will be the same, only the `credentials` value to be encoded differs.
Instead of repeating the block for a similar route, this difference is handled via the Caddyfile `map` directive.
We match the path context (_`/xoauth2` or `/oauthbearer`, the `/imap` prefix was stripped by `handle_path` earlier_), when there is a valid match, `sasl_mechanism` and `credentials` map vars are created and assigned to be referenced by the later `respond` directive.
---
Repeat the same test-case logic, DRY with log asserts extracted to a common function call. This should be fine as the auth method will be sufficient to match against or a common failure caught.
* tests: OAuth2 - Minor revisions
Separate test cases and additional comment on creating the same base64 encoded credentials via CLI as an alternative to running Caddy.
Added a simple `compose.yaml` for troubleshooting or running the container for the `/imap/xoauth2` / `/imap/oauthbearer` endpoints.
* tests: OAuth2 - Route endpoints in Caddyfile with snippets instead
`reverse_proxy` was a bit more convenient, but the additional internal ports weren't really relevant. It also added noise to logging when troubleshooting.
The `import` directive with Snippet blocks instead is a bit cleaner, but when used in a single file snippets must be defined prior to referencing them with the `import` directive.
---
`compose.yaml` inlines the examples, with slight modification to `localhost:80`, since the Caddyfile examples `auth.example.test` is more relevant to the tests which can use it, and not applicable to troubleshooting locally outside of tests.
* chore: Add entry to `CHANGELOG.md`
* chore: Additional context on access token
