| author | Alberto Bertogli
<albertito@blitiri.com.ar> 2026-10-14 14:14:42 UTC |
| committer | Alberto Bertogli
<albertito@blitiri.com.ar> 2026-10-14 14:19:34 UTC |
| parent | 6cc59ca6611ba241410b68764be4d12cde94fd57 |
| .mkdocs.yml | +2 | -0 |
| Makefile | +4 | -1 |
| README.md | +39 | -45 |
| docker/README.md | +12 | -16 |
| docs/aliases.md | +16 | -23 |
| docs/clients.md | +4 | -10 |
| docs/contributing.md | +14 | -20 |
| docs/dkim.md | +21 | -29 |
| docs/dovecot.md | +3 | -8 |
| docs/flow.md | +22 | -27 |
| docs/haproxy.md | +1 | -5 |
| docs/hooks.md | +27 | -33 |
| docs/howto.md | +50 | -65 |
| docs/index.md | +3 | -8 |
| docs/install.md | +21 | -33 |
| docs/knownissues.md | +2 | -5 |
| docs/man/index.md | +3 | -5 |
| docs/monitoring.md | +45 | -48 |
| docs/relnotes.md | +19 | -47 |
| docs/sec-levels.md | +27 | -34 |
| docs/security.md | +1 | -2 |
| test/README.md | +21 | -31 |
diff --git a/.mkdocs.yml b/.mkdocs.yml index a557001..b443411 100644 --- a/.mkdocs.yml +++ b/.mkdocs.yml @@ -14,6 +14,8 @@ markdown_extensions: guess_lang: false - attr_list - admonition + - pymdownx.escapeall: + hardbreak: true theme: readthedocs diff --git a/Makefile b/Makefile index f6bfc1e..b3ed6a9 100644 --- a/Makefile +++ b/Makefile @@ -60,10 +60,13 @@ fmt: gofmt -s -w . go fix ./... clang-format -i $(shell find . -iname '*.proto') - black -l 79 \ + black -l 79 --quiet \ test/util/check-hostaliases \ test/util/chamuyero \ test/util/mail_diff + mdformat --wrap 79 --number \ + --exclude 'docs/man/*.[15].md' \ + README.md docs/ .PHONY: chasquid test \ chasquid-util smtp-check mda-lmtp dovecot-auth-cli \ diff --git a/README.md b/README.md index 34e3cc0..d27a68a 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,3 @@ - # chasquid [chasquid](https://blitiri.com.ar/p/chasquid) is an SMTP (email) server with a @@ -11,64 +10,45 @@ It's written in [Go](https://golang.org), and distributed under the [](https://github.com/albertito/chasquid/actions) [](https://goreportcard.com/report/github.com/albertito/chasquid) -[](https://coveralls.io/github/albertito/chasquid?branch=next) +[](https://coveralls.io/github/albertito/chasquid?branch=next)\ [](https://blitiri.com.ar/p/chasquid/) [](https://webchat.oftc.net/?channels=%23chasquid) - ## Features -* Easy - * Easy to configure. - * Hard to mis-configure in ways that are harmful or insecure (e.g. no open - relay, or clear-text authentication). - * [Monitoring] HTTP server, with exported variables and tracing to help - debugging. - * Integrated with [Debian], [Ubuntu], [Arch], and [Alpine]. - * Supports using [Dovecot] for authentication. -* Useful - * Multiple/virtual domains, with per-domain users and aliases. - * Suffix dropping (`user+something@domain` → `user@domain`). - * [Hooks] for integration with greylisting, anti-virus, and anti-spam. - * International usernames ([SMTPUTF8]) and domain names ([IDNA]). -* Secure - * [Tracking] of per-domain TLS support, prevents connection downgrading. - * Multiple TLS certificates. - * Easy integration with [Let's Encrypt]. - * [SPF] and [MTA-STS] checking. - * [DKIM] support (signing and verification). - - -[Arch]: https://blitiri.com.ar/p/chasquid/install/#arch -[Alpine]: https://blitiri.com.ar/p/chasquid/install/#alpine -[DKIM]: https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail -[Debian]: https://blitiri.com.ar/p/chasquid/install/#debianubuntu -[Dovecot]: https://blitiri.com.ar/p/chasquid/dovecot/ -[Hooks]: https://blitiri.com.ar/p/chasquid/hooks/ -[IDNA]: https://en.wikipedia.org/wiki/Internationalized_domain_name -[Let's Encrypt]: https://letsencrypt.org -[MTA-STS]: https://tools.ietf.org/html/rfc8461 -[Monitoring]: https://blitiri.com.ar/p/chasquid/monitoring/ -[SMTPUTF8]: https://en.wikipedia.org/wiki/Extended_SMTP#SMTPUTF8 -[SPF]: https://en.wikipedia.org/wiki/Sender_Policy_Framework -[Tracking]: https://blitiri.com.ar/p/chasquid/sec-levels/ -[Ubuntu]: https://blitiri.com.ar/p/chasquid/install/#debianubuntu - +- Easy + - Easy to configure. + - Hard to mis-configure in ways that are harmful or insecure (e.g. no open + relay, or clear-text authentication). + - [Monitoring] HTTP server, with exported variables and tracing to help + debugging. + - Integrated with [Debian], [Ubuntu], [Arch], and [Alpine]. + - Supports using [Dovecot] for authentication. +- Useful + - Multiple/virtual domains, with per-domain users and aliases. + - Suffix dropping (`user+something@domain` → `user@domain`). + - [Hooks] for integration with greylisting, anti-virus, and anti-spam. + - International usernames ([SMTPUTF8]) and domain names ([IDNA]). +- Secure + - [Tracking] of per-domain TLS support, prevents connection downgrading. + - Multiple TLS certificates. + - Easy integration with [Let's Encrypt]. + - [SPF] and [MTA-STS] checking. + - [DKIM] support (signing and verification). ## Documentation The [how-to guide](https://blitiri.com.ar/p/chasquid/howto/) and the -[installation guide](https://blitiri.com.ar/p/chasquid/install/) are the -best starting points on how to install, configure and run chasquid. +[installation guide](https://blitiri.com.ar/p/chasquid/install/) are the best +starting points on how to install, configure and run chasquid. You will find [all documentation here](https://blitiri.com.ar/p/chasquid/). - ## Contact -If you have any questions, comments or patches, please send them to the [mailing -list](https://groups.google.com/forum/#!forum/chasquid), -chasquid@googlegroups.com. +If you have any questions, comments or patches, please send them to the +[mailing list](https://groups.google.com/forum/#!forum/chasquid), +chasquid@googlegroups.com.\ To subscribe, send an email to chasquid+subscribe@googlegroups.com. Security issues can be reported privately to albertito@blitiri.com.ar. @@ -78,3 +58,17 @@ Bug reports and pull requests on You can also reach out via IRC, `#chasquid` on [OFTC](https://oftc.net/). +[alpine]: https://blitiri.com.ar/p/chasquid/install/#alpine +[arch]: https://blitiri.com.ar/p/chasquid/install/#arch +[debian]: https://blitiri.com.ar/p/chasquid/install/#debianubuntu +[dkim]: https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail +[dovecot]: https://blitiri.com.ar/p/chasquid/dovecot/ +[hooks]: https://blitiri.com.ar/p/chasquid/hooks/ +[idna]: https://en.wikipedia.org/wiki/Internationalized_domain_name +[let's encrypt]: https://letsencrypt.org +[monitoring]: https://blitiri.com.ar/p/chasquid/monitoring/ +[mta-sts]: https://tools.ietf.org/html/rfc8461 +[smtputf8]: https://en.wikipedia.org/wiki/Extended_SMTP#SMTPUTF8 +[spf]: https://en.wikipedia.org/wiki/Sender_Policy_Framework +[tracking]: https://blitiri.com.ar/p/chasquid/sec-levels/ +[ubuntu]: https://blitiri.com.ar/p/chasquid/install/#debianubuntu diff --git a/docker/README.md b/docker/README.md index f704264..e2af891 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,4 +1,3 @@ - # Docker chasquid comes with a Dockerfile to create a container running [chasquid], @@ -7,18 +6,13 @@ chasquid comes with a Dockerfile to create a container running [chasquid], Note these are less thoroughly tested than the [traditional setup](howto.md), which is the recommended way to use chasquid. -[chasquid]: https://blitiri.com.ar/p/chasquid -[dovecot]: https://dovecot.org -[Let's Encrypt]: https://letsencrypt.org - - ## Images There are pre-built images at the -[gitlab registry](https://gitlab.com/albertito/chasquid/container_registry) -and [dockerhub](https://hub.docker.com/r/albertito/chasquid). -They are automatically built, and tagged with the corresponding branch name. -Use the *main* tag for a stable version. +[gitlab registry](https://gitlab.com/albertito/chasquid/container_registry) and +[dockerhub](https://hub.docker.com/r/albertito/chasquid). They are +automatically built, and tagged with the corresponding branch name. Use the +*main* tag for a stable version. If, instead, you want to build the image yourself, just run: @@ -59,12 +53,12 @@ Password: pepe@example.com added to /data/dovecot/users ``` -Upon startup, the image will obtain a TLS certificate for you using [Let's -Encrypt](https://letsencrypt.com/). You need to tell it the domain(s) to get a -certificate from by setting the `AUTO_CERTS` variable. +Upon startup, the image will obtain a TLS certificate for you using +[Let's Encrypt](https://letsencrypt.com/). You need to tell it the domain(s) to +get a certificate from by setting the `AUTO_CERTS` variable. -Because certificates expire, you should restart the container every week or -so. Certificates will be renewed automatically upon startup if needed. +Because certificates expire, you should restart the container every week or so. +Certificates will be renewed automatically upon startup if needed. In order for chasquid to get access to the source IP address, you will need to use host networking, or create a custom docker network that does IP forwarding @@ -79,10 +73,12 @@ $ docker run -e AUTO_CERTS=mail.yourdomain.com \ registry.gitlab.com/albertito/chasquid:main ``` - ## Debugging To get a shell on the running container for debugging, you can use `docker ps` to find the container ID, and then `docker exec -it CONTAINERID /bin/bash` to open a shell on the running container. +[chasquid]: https://blitiri.com.ar/p/chasquid +[dovecot]: https://dovecot.org +[let's encrypt]: https://letsencrypt.org diff --git a/docs/aliases.md b/docs/aliases.md index 568ac79..a90ba22 100644 --- a/docs/aliases.md +++ b/docs/aliases.md @@ -1,9 +1,7 @@ - # Aliases -[chasquid] supports [email aliases], which is a mechanism to redirect mail -from one account to others. - +[chasquid] supports [email aliases], which is a mechanism to redirect mail from +one account to others. ## File format @@ -42,8 +40,8 @@ email will be forwarded using While the content of the message will not be changed, the envelope sender will be the constructed from the alias user. -User names cannot contain spaces, ":" or commas, for parsing reasons. This is -a tradeoff between flexibility and keeping the file format easy to edit for +User names cannot contain spaces, ":" or commas, for parsing reasons. This is a +tradeoff between flexibility and keeping the file format easy to edit for people. User names will be normalized internally to lower-case. UTF-8 is allowed and fully supported. @@ -52,8 +50,8 @@ allowed and fully supported. A pipe alias is of the form `user: | command`, and causes mail to be sent as standard input to the given command. -The command can have space-separated arguments (no quotes or escaping -expansion will be performed). +The command can have space-separated arguments (no quotes or escaping expansion +will be performed). For example: @@ -76,13 +74,12 @@ pepe: jose *: pepe, rose@backgarden ``` -If the destination address has `*` as its user, then it will be replaced -by the sender user. Note that in this case, the user is copied as-is, no -characters or suffixes will be dropped. +If the destination address has `*` as its user, then it will be replaced by the +sender user. Note that in this case, the user is copied as-is, no characters or +suffixes will be dropped. For example, `*: *@pond` will redirect `lilly@domain` to `lilly@pond`. - ### "Via" aliases {#via} A "via" alias is like an email alias, but it explicitly specifies which @@ -111,7 +108,6 @@ flowers: lilly@pond via mail1.pond/mail2.pond *: * via mail1.example.com ``` - ### Overrides If the same left-side address appears more than once, the last one will take @@ -156,31 +152,29 @@ juana.perez+fruta -> fruta juanaperez+fruta -> fruta ``` -This allows addresses with suffixes to have specific aliases, without having -to worry about drop characters, which is the most common use case. +This allows addresses with suffixes to have specific aliases, without having to +worry about drop characters, which is the most common use case. If different semantics are needed, they can be implemented using the [hook](#hooks). - ## Processing -Aliases files are read upon start-up and refreshed every 30 seconds, so -changes to them don't require a daemon restart. +Aliases files are read upon start-up and refreshed every 30 seconds, so changes +to them don't require a daemon restart. The resolver will perform lookups recursively, until it finds all the final recipients. There are recursion limits to avoid alias loops. If the limit (10 levels) is reached, the entire resolution will fail. Commands are given 30s to run, after which it will be killed and the execution -will fail. If the command exits with an error (non-0 exit code), the delivery +will fail. If the command exits with an error (non-0 exit code), the delivery will be considered failed. -The `chasquid-util` command-line tool can be used to check and resolve -aliases manually. It talks to the running server, so the response is fully +The `chasquid-util` command-line tool can be used to check and resolve aliases +manually. It talks to the running server, so the response is fully authoritative. - ## Hooks There is a hook that allows more sophisticated aliases resolution: @@ -191,6 +185,5 @@ are merged with the file-based resolution results. See the [hooks](hooks.md) documentation for more details. - [chasquid]: https://blitiri.com.ar/p/chasquid [email aliases]: https://en.wikipedia.org/wiki/Email_alias diff --git a/docs/clients.md b/docs/clients.md index 1d7bc2b..74dd311 100644 --- a/docs/clients.md +++ b/docs/clients.md @@ -1,18 +1,15 @@ - # Clients chasquid supports most SMTP clients, but requires them to have some features: -- Support TLS (either - [STARTTLS](https://datatracker.ietf.org/doc/html/rfc3207) or - [implicit TLS](https://datatracker.ietf.org/doc/html/rfc8314#section-3.3)) +- Support TLS (either [STARTTLS](https://datatracker.ietf.org/doc/html/rfc3207) + or [implicit TLS](https://datatracker.ietf.org/doc/html/rfc8314#section-3.3)) - Support the [PLAIN authentication method](https://datatracker.ietf.org/doc/html/rfc4954#section-4). All modern clients should support both, and thus have no problems talking to chasquid. - ## Configuration examples ### [msmtp](https://marlam.de/msmtp/) @@ -42,13 +39,10 @@ password SECRET Replace the `SERVER`, `USER@DOMAIN` and `SECRET` strings with the appropriate values. - ## Problematic clients These clients are known to have issues talking to chasquid: -- [ssmtp](https://packages.debian.org/source/unstable/ssmtp): does not - support the PLAIN authentication method. It is also unmaintained. +- [ssmtp](https://packages.debian.org/source/unstable/ssmtp): does not support + the PLAIN authentication method. It is also unmaintained.\ Please use [msmtp](https://marlam.de/msmtp/) instead. - - diff --git a/docs/contributing.md b/docs/contributing.md index ff3f2ed..bfd61d7 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -1,19 +1,17 @@ - # Contributing ## Questions and bug reports -Please send questions and bug reports via [Github Issues], or to the [mailing -list], chasquid@googlegroups.com. +Please send questions and bug reports via [Github Issues], or to the +[mailing list], chasquid@googlegroups.com.\ To subscribe to the [mailing list], send an email to `chasquid+subscribe@googlegroups.com`. -To privately report a suspected security issue, please see [Reporting a -security issue](security.md). - -<a name="irc"></a> -You can also reach out via IRC, `#chasquid` on [OFTC](https://oftc.net/). +To privately report a suspected security issue, please see +[Reporting a security issue](security.md). +<a name="irc"></a> You can also reach out via IRC, `#chasquid` on +[OFTC](https://oftc.net/). ## Patches and pull requests @@ -24,12 +22,11 @@ Before sending any non-trivial change, it's ideal to discuss the proposal (via [Github Issues], [mailing list], or [IRC](#irc)). This helps make the most out of the contribution, and minimize friction frustration during code reviews. - ### Commit message, coding style, tests Ideally, patches would have descriptive commit messages, adhere to Go's coding -style, and include comprehensive tests. However, that is a lot to ask, and -very subjective. +style, and include comprehensive tests. However, that is a lot to ask, and very +subjective. It is okay to propose patches without those things. In those cases, the maintainer will usually amend the patches to add them. @@ -39,22 +36,19 @@ For how to write commit messages, [this](https://git-scm.com/docs/SubmittingPatches#describe-changes) articles contain great advise. - ### Workflow Patches will be cherry-picked, and are typically first incorporated into the `next` branch. -The `next` branch is where patches are staged, tested and reviewed. This -branch is rebased frequently, to make adjustments and fixes as needed. -Once patches have received enough testing (usually after a couple of weeks), -they are moved to the `main` branch. +The `next` branch is where patches are staged, tested and reviewed. This branch +is rebased frequently, to make adjustments and fixes as needed. Once patches +have received enough testing (usually after a couple of weeks), they are moved +to the `main` branch. The `main` branch is the stable branch and where releases are cut from. It is never rebased. - -[Github Issues]: https://github.com/albertito/chasquid/issues +[github issues]: https://github.com/albertito/chasquid/issues +[github pull requests]: https://github.com/albertito/chasquid/pulls [mailing list]: https://groups.google.com/forum/#!forum/chasquid -[GitHub's Security tab]: https://github.com/albertito/chasquid/security -[Github Pull requests]: https://github.com/albertito/chasquid/pulls diff --git a/docs/dkim.md b/docs/dkim.md index e48e2c0..fbde985 100644 --- a/docs/dkim.md +++ b/docs/dkim.md @@ -1,4 +1,3 @@ - # DKIM integration [chasquid] supports verifying and generating [DKIM] signatures since version @@ -9,25 +8,22 @@ have a private DKIM key set up will be signed. In versions older than 1.13, support is possible via the [hooks] mechanism. In particular, the [example hook] included support for some command-line -implementations. That continues to be an option, especially if customization -is needed. - +implementations. That continues to be an option, especially if customization is +needed. ## Easy setup - Run `chasquid-util dkim-keygen DOMAIN` to generate a DKIM private key for your domain. The file will be in `/etc/chasquid/domains/DOMAIN/dkim:*.pem`. -- Publish the DKIM DNS record which was shown by the - previous command (e.g. by following - [this guide](https://support.dnsimple.com/articles/dkim-record/)). +- Publish the DKIM DNS record which was shown by the previous command (e.g. by + following [this guide](https://support.dnsimple.com/articles/dkim-record/)). - Change the key file's permissions, to ensure it is readable by chasquid (and nobody else). - Restart chasquid. It is highly recommended that you use a DKIM checker (like -[Learn DMARC](https://www.learndmarc.com/)) to confirm that your setup is -fully functional. - +[Learn DMARC](https://www.learndmarc.com/)) to confirm that your setup is fully +functional. ## Advanced setup @@ -38,16 +34,15 @@ It needs to be either RSA or Ed25519. ### Key rotation -To rotate a key, you can remove the old key file, and generate a new one as -per the previous step. - -It is important to remove the old key from the directory, because chasquid -will use *all* the keys in it. +To rotate a key, you can remove the old key file, and generate a new one as per +the previous step. -You should use a different selector each time. If you don't specify a -selector when using `chasquid-util dkim-keygen`, the current date will be -used, which is a safe default to prevent accidental reuse. +It is important to remove the old key from the directory, because chasquid will +use *all* the keys in it. +You should use a different selector each time. If you don't specify a selector +when using `chasquid-util dkim-keygen`, the current date will be used, which is +a safe default to prevent accidental reuse. ### Multiple keys @@ -57,18 +52,16 @@ support multiple signing algorithms). This is well supported: chasquid will sign email with all keys it find that match `dkim:*.pem` in a domain directory. - ## Verification [chasquid] will verify all DKIM signatures of incoming mail, and record the results in an [`Authentication-Results:`] header, as per [RFC 8601]. -Note that emails will *not* be rejected even if they fail verification, as -this is not recommended +Note that emails will *not* be rejected even if they fail verification, as this +is not recommended ([source 1](https://tools.ietf.org/html/rfc6376#section-6.3), [source 2](https://tools.ietf.org/html/rfc7601#section-2.7.1)). - ## Other implementations [chasquid] also supports [DKIM] via the [hooks] mechanism. This can be useful @@ -79,12 +72,11 @@ Implementations that have been tried: - [driusan/dkim] - [dkimpy] - [chasquid]: https://blitiri.com.ar/p/chasquid -[DKIM]: https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail -[hooks]: hooks.md -[example hook]: https://blitiri.com.ar/git/r/chasquid/b/next/t/etc/chasquid/hooks/f=post-data.html -[driusan/dkim]: https://github.com/driusan/dkim +[dkim]: https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail [dkimpy]: https://launchpad.net/dkimpy/ -[RFC 8601]: https://datatracker.ietf.org/doc/html/rfc8601 -[`Authentication-Results:`]: https://en.wikipedia.org/wiki/Email_authentication#Authentication-Results +[driusan/dkim]: https://github.com/driusan/dkim +[example hook]: https://blitiri.com.ar/git/r/chasquid/b/next/t/etc/chasquid/hooks/f=post-data.html +[hooks]: hooks.md +[rfc 8601]: https://datatracker.ietf.org/doc/html/rfc8601 +[`authentication-results:`]: https://en.wikipedia.org/wiki/Email_authentication#Authentication-Results diff --git a/docs/dovecot.md b/docs/dovecot.md index 660c9de..1700ec8 100644 --- a/docs/dovecot.md +++ b/docs/dovecot.md @@ -1,4 +1,3 @@ - # Dovecot integration As of version 0.04 (2018-02), [chasquid] has integration with [dovecot] for @@ -7,7 +6,6 @@ authenticating users. This means that chasquid can ask dovecot to authenticate users, instead/in addition to having its own per-domain user databases. - ## Configuring dovecot The following needs to be added to the Dovecot configuration, usually in @@ -35,7 +33,6 @@ Authentication requests sent by chasquid to dovecot will pass on the username as specified by the client. This will usually be either `user@domain`, or just `user`. - ## Configuring chasquid Add the following line to `/etc/chasquid/chasquid.conf`: @@ -51,14 +48,13 @@ CentOS, and NixOS). If chasquid can't find them, the paths can be set with the `dovecot_userdb_path` and `dovecot_client_path` options. - ## Troubleshooting Dovecot authentication can be tricky to troubleshoot. If you think it is not working as it should, or chasquid isn't correctly -talking with it, the easiest way to check is to [increase dovecot auth logging -verbosity](https://doc.dovecot.org/admin_manual/logging/?highlight=logging#logging-verbosity): +talking with it, the easiest way to check is to +[increase dovecot auth logging verbosity](https://doc.dovecot.org/admin_manual/logging/?highlight=logging#logging-verbosity): ``` auth_verbose = yes @@ -69,6 +65,5 @@ One common gotcha is when dovecot is set up to use `user` instead of `user@domain`. In that case you can try setting `auth_username_format = %n` to make it ignore the domain if present. - -[dovecot]: https://dovecot.org [chasquid]: https://blitiri.com.ar/p/chasquid +[dovecot]: https://dovecot.org diff --git a/docs/flow.md b/docs/flow.md index 078dbec..ea84fa6 100644 --- a/docs/flow.md +++ b/docs/flow.md @@ -1,34 +1,31 @@ - # Message flows This document explains at a high level some parts of chasquid's message processing, in particular how messages flow through the system. - ## Message reception - Client connects to chasquid on the smtp or submission ports, and issues HELO/EHLO. - Client optionally performs STARTTLS. -- Client optionally performs AUTH. - - Check that this is done over TLS. -- Client sends MAIL FROM. - - Check SPF. - - Check connection security level. -- Client sends one or more RCPT TO. - - If the destination is remote, then the user must have authenticated. - - If the destination is local, check that the user exists. +- Client optionally performs AUTH, which must be done over TLS. +- Client sends MAIL FROM, and we check SPF and the connection security level. +- Client sends one or more RCPT TO. If the destination is remote, then the user + must have authenticated. If the destination is local, check that the user + exists. - Client sends DATA. - Client sends actual data, and ends it with '.' - - Parse the data contents to perform loop detection. - - If the sender is authenticated, DKIM-sign the email with the - corresponding key. - - If the sender is not authenticated, verify the DKIM signature (if the - email has one). - - Add the required headers (Received, SPF results, post-data hook output). - - Run the post-data hook. If the hook fails, return an error. - - Put it in the queue and reply success. +Then, to process the data: + +- Parse the data contents to perform loop detection. +- If the sender is authenticated, DKIM-sign the email with the corresponding + key. +- If the sender is not authenticated, verify the DKIM signature (if the email + has one). +- Add the required headers (Received, SPF results, post-data hook output). +- Run the post-data hook. If the hook fails, return an error. +- Put it in the queue and reply success. ### Authenticated mail, and email spoofing @@ -41,9 +38,9 @@ Users who want to be strict about "MAIL FROM" or even "From:" validation can add additional checks in the [post-DATA hook](hooks.md). In the future, chasquid may get some option to be strict about it by default, -or on a per-domain or per-user basis. But for now, using a [post-DATA -hook](hooks.md) is the best way to make chasquid more strict about this. - +or on a per-domain or per-user basis. But for now, using a +[post-DATA hook](hooks.md) is the best way to make chasquid more strict about +this. ## Queue processing @@ -57,11 +54,9 @@ Before accepting a message: Queue processing runs asynchronously, there's a goroutine for each message which does, in a loop: -- For each recipient which we have not delivered yet: - - Attempt delivery. - - Write to disk the results. +- For each recipient which we have not delivered yet, attempt delivery, and + write the results to disk. - If there are mails still pending, wait for some time (incrementally). - When all the recipients have completed delivery, or enough time has passed: - - If all were successful, remove from the queue. - - If some failed, send a delivery status notification back to the sender. - + if all were successful, remove from the queue; if some failed, send a + delivery status notification back to the sender. diff --git a/docs/haproxy.md b/docs/haproxy.md index 57740f6..5d6a9f5 100644 --- a/docs/haproxy.md +++ b/docs/haproxy.md @@ -1,10 +1,8 @@ - # HAProxy integration As of version 1.6, [chasquid] supports being deployed behind a [HAProxy] instance. - ## Configuring HAProxy In the backend server line, set the [send-proxy] parameter to turn on the use @@ -14,7 +12,6 @@ You need to set this for each of the ports that are forwarded. Only PROXY protocol v1 is supported for now. - ## Configuring chasquid Add the following line to `/etc/chasquid/chasquid.conf`: @@ -25,7 +22,6 @@ haproxy_incoming: true That turns HAProxy support on for all incoming SMTP connections. - [chasquid]: https://blitiri.com.ar/p/chasquid -[HAProxy]: https://www.haproxy.org/ +[haproxy]: https://www.haproxy.org/ [send-proxy]: http://cbonte.github.io/haproxy-dconv/2.0/configuration.html#5.2-send-proxy diff --git a/docs/hooks.md b/docs/hooks.md index bf8a7d1..a14371d 100644 --- a/docs/hooks.md +++ b/docs/hooks.md @@ -1,4 +1,3 @@ - # Hooks chasquid supports some functionality via hooks, which are binaries that get @@ -6,7 +5,6 @@ executed at specific points in time during delivery. They are optional, and will be skipped if they don't exist. - ## Post-DATA hook After completion of DATA, but before accepting the mail for queueing, chasquid @@ -15,19 +13,16 @@ will run the command at `$config_dir/hooks/post-data`. The contents of the mail will be written to the command's stdin, and the environment is detailed below. -If the exit status is 0, chasquid will move forward processing the command, -and its stdout should contain headers which will be added to contents of -the email (at the top). +If the exit status is 0, chasquid will move forward processing the command, and +its stdout should contain headers which will be added to contents of the email +(at the top). Otherwise, chasquid will respond with an error, and the last line of stdout -will be passed back to the client as the error message. -If the exit status is 20 the error code will be permanent, otherwise it will -be temporary. - - -This hook can be used to block based on contents, for example to check for -spam or virus. See `etc/hooks/post-data` for an example. +will be passed back to the client as the error message. If the exit status is +20 the error code will be permanent, otherwise it will be temporary. +This hook can be used to block based on contents, for example to check for spam +or virus. See `etc/hooks/post-data` for an example. ### Environment @@ -36,27 +31,26 @@ privileges. The environment will contain the following variables: - - `$USER`: The server's `$USER` env variable. - - `$SHELL`: The server's `$SHELL` env variable. - - `$PATH`: The server's `$PATH` env variable. - - `$PWD`: The working directory, which will be the config directory. - - `$REMOTE_ADDR`: IP address of the remote side of the connection. - - `$EHLO_DOMAIN`: EHLO/HELO domain, as given by the client; sanitized for - safety (only letters, digits, and `-.[]:` are kept). If there are no valid - characters, it is `no-valid-characters.invalid`. - - `$EHLO_DOMAIN_RAW`: Same as `$EHLO_DOMAIN`. Kept for backwards - compatibility; it used to be the unsanitized value. - - `$MAIL_FROM`: MAIL FROM address. - - `$RCPT_TO`: RCPT TO addresses, space separated. - - `$AUTH_AS`: Authenticated user; empty if the connection has not - authenticated successfully. - - `$ON_TLS`: 1 if using TLS, 0 if not. - - `$FROM_LOCAL_DOMAIN`: 1 if the mail comes from a local domain, 0 if not. - - `$SPF_PASS`: 1 if it passed SPF, 0 if not. - -There is a 1 minute timeout for hook execution. -It will be run at the config directory. - +- `$USER`: The server's `$USER` env variable. +- `$SHELL`: The server's `$SHELL` env variable. +- `$PATH`: The server's `$PATH` env variable. +- `$PWD`: The working directory, which will be the config directory. +- `$REMOTE_ADDR`: IP address of the remote side of the connection. +- `$EHLO_DOMAIN`: EHLO/HELO domain, as given by the client; sanitized for + safety (only letters, digits, and `-.[]:` are kept). If there are no valid + characters, it is `no-valid-characters.invalid`. +- `$EHLO_DOMAIN_RAW`: Same as `$EHLO_DOMAIN`. Kept for backwards compatibility; + it used to be the unsanitized value. +- `$MAIL_FROM`: MAIL FROM address. +- `$RCPT_TO`: RCPT TO addresses, space separated. +- `$AUTH_AS`: Authenticated user; empty if the connection has not authenticated + successfully. +- `$ON_TLS`: 1 if using TLS, 0 if not. +- `$FROM_LOCAL_DOMAIN`: 1 if the mail comes from a local domain, 0 if not. +- `$SPF_PASS`: 1 if it passed SPF, 0 if not. + +There is a 1 minute timeout for hook execution. It will be run at the config +directory. ## Alias resolve hook diff --git a/docs/howto.md b/docs/howto.md index f631b6b..bc07453 100644 --- a/docs/howto.md +++ b/docs/howto.md @@ -1,4 +1,3 @@ - # chasquid how-to guide This is a practical guide for setting up an email server for personal or small @@ -7,34 +6,22 @@ detailed references where possible. While a lot of the contents are generic, for simplicity it will use: - - [Debian] as base operating system ([Ubuntu] also works) - - [Dovecot] for [POP3]+[IMAP] - - [chasquid] for [SMTP] - - [Let's Encrypt] for [TLS] certificates - -[Debian]: https://debian.org -[Ubuntu]: https://ubuntu.com -[Dovecot]: https://dovecot.org -[chasquid]: https://blitiri.com.ar/p/chasquid -[Let's Encrypt]: https://letsencrypt.org -[POP3]: https://en.wikipedia.org/wiki/Post_Office_Protocol -[IMAP]: https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol -[SMTP]: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol -[TLS]: https://en.wikipedia.org/wiki/Transport_Layer_Security - +- [Debian] as base operating system ([Ubuntu] also works) +- [Dovecot] for [POP3]+[IMAP] +- [chasquid] for [SMTP] +- [Let's Encrypt] for [TLS] certificates ## Example data This guide will use the following data for illustration purposes, replace them with your own where appropriate. - - Domain name: `example.com`. - - IPv4 address of your mail server: `198.51.100.7`. - - IPv6 address of your mail server: `2001:db8::7`. +- Domain name: `example.com`. +- IPv4 address of your mail server: `198.51.100.7`. +- IPv6 address of your mail server: `2001:db8::7`. Note IPv6 is optional but highly encouraged, and supported by most providers. - ## Getting a server You first need to have a server to use. This could be an existing one (for @@ -43,33 +30,26 @@ exclusive for email. In this guide we will use a separate server, mostly for clarity. -For small groups the size of the server does not matter, any small VPS -(virtual private server) will do just fine. +For small groups the size of the server does not matter, any small VPS (virtual +private server) will do just fine. Specifically for hosting email servers, there are some things to check when selecting a provider: - - Make sure they allow traffic on TCP port 25 (SMTP). While almost all VPS - and dedicated server providers are fine, some "cloud" providers (like - Google Cloud) block port 25, which is used for sending and receiving mails. - - Once you get a server, make sure the IP addresses are not listed in any - [blackhole lists]. - There are many services to check them, for example [the - Anti-Abuse project] or [multirbl.valli.org]. +- Make sure they allow traffic on TCP port 25 (SMTP). While almost all VPS and + dedicated server providers are fine, some "cloud" providers (like Google + Cloud) block port 25, which is used for sending and receiving mails. +- Once you get a server, make sure the IP addresses are not listed in any + [blackhole lists]. There are many services to check them, for example + [the Anti-Abuse project] or [multirbl.valli.org]. Remember to update your server regularly, setting up [unattended upgrades] is highly recommended. -[the Anti-Abuse project]: http://www.anti-abuse.org/multi-rbl-check/ -[blackhole lists]: https://en.wikipedia.org/wiki/DNSBL -[unattended upgrades]: https://wiki.debian.org/UnattendedUpgrades -[multirbl.valli.org]: http://multirbl.valli.org/ - - ## DNS -Set up the following DNS records for `example.com`. This is usually done -either in your DNS server, or in the user interface of your DNS provider. +Set up the following DNS records for `example.com`. This is usually done either +in your DNS server, or in the user interface of your DNS provider. ``` ; Assign "mail.example.com" to the server's IP addresses. @@ -86,20 +66,18 @@ mail AAAA 2001:db8::7 ``` Finally, you should go to your server provider and configure the "reverse DNS" -(also known as "PTR") for the IP addresses to be to "mail.example.com". This -is important, as some spam checkers will consider it a factor. +(also known as "PTR") for the IP addresses to be to "mail.example.com". This is +important, as some spam checkers will consider it a factor. -*References: -[A record](https://en.wikipedia.org/wiki/A_record), +*References: [A record](https://en.wikipedia.org/wiki/A_record), [MX record](https://en.wikipedia.org/wiki/MX_record), [Sender Policy Framework (SPF)](https://en.wikipedia.org/wiki/Sender_Policy_Framework).* - ## TLS certificate -[TLS] certificates are needed to send and receive email securely. -[letsencrypt] will provide us with a free certificate, which needs to be -renewed every 90 days, so the following relies on automatic renewal. +[TLS] certificates are needed to send and receive email securely. [letsencrypt] +will provide us with a free certificate, which needs to be renewed every 90 +days, so the following relies on automatic renewal. Note `certbot` is the recommended letsencrypt command line client. @@ -124,10 +102,6 @@ EOF sudo chmod +x /etc/letsencrypt/renewal-hooks/post/restart ``` -[TLS]: https://en.wikipedia.org/wiki/Transport_Layer_Security -[letsencrypt]: https://letsencrypt.org - - ## Configure dovecot First, install dovecot, and let chasquid use it for authorizing users. That @@ -151,12 +125,11 @@ service auth { EOF ``` -You will need to configure dovecot authentication depending on your needs. -For example, if you want to use only system users, or virtual users. -See the `/etc/dovecot/conf.d/10-auth.conf` file, and the [dovecot -documentation](https://wiki.dovecot.org/HowTo/SimpleVirtualInstall) for more -details. - +You will need to configure dovecot authentication depending on your needs. For +example, if you want to use only system users, or virtual users. See the +`/etc/dovecot/conf.d/10-auth.conf` file, and the +[dovecot documentation](https://wiki.dovecot.org/HowTo/SimpleVirtualInstall) +for more details. ## Configure chasquid @@ -191,13 +164,12 @@ EOF ## Add users -With this configuration, chasquid will use dovecot to manage users, so refer -to the [dovecot documentation](https://wiki.dovecot.org/BasicConfiguration) -for the details. +With this configuration, chasquid will use dovecot to manage users, so refer to +the [dovecot documentation](https://wiki.dovecot.org/BasicConfiguration) for +the details. You can also add chasquid-specific users with `chasquid-util user-add`. - ## Additional domains To make chasquid manage an additional domain `otherdomain.com`, first add the @@ -208,13 +180,13 @@ following records to `otherdomain.com`: @ TXT "v=spf1 mx -all" ``` -Then, tell chasquid about it by running `mkdir -/etc/chasquid/domains/otherdomain.com`. Don't forget to restart it afterwards. +Then, tell chasquid about it by running +`mkdir /etc/chasquid/domains/otherdomain.com`. Don't forget to restart it +afterwards. Alternatively, you can use a different MX record, as long as you can get chasquid a certificate for it. - ## Optional software - To use [SpamAssassin] to filter spam, run `apt install spamassassin spamc`. @@ -223,6 +195,19 @@ chasquid a certificate for it. That's all it takes. chasquid default hooks will pass incoming mail through both if (and only if) they are installed. -[SpamAssassin]: https://spamassassin.apache.org/ -[ClamAV]: https://www.clamav.net/ - +[blackhole lists]: https://en.wikipedia.org/wiki/DNSBL +[chasquid]: https://blitiri.com.ar/p/chasquid +[clamav]: https://www.clamav.net/ +[debian]: https://debian.org +[dovecot]: https://dovecot.org +[imap]: https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol +[let's encrypt]: https://letsencrypt.org +[letsencrypt]: https://letsencrypt.org +[multirbl.valli.org]: http://multirbl.valli.org/ +[pop3]: https://en.wikipedia.org/wiki/Post_Office_Protocol +[smtp]: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol +[spamassassin]: https://spamassassin.apache.org/ +[the anti-abuse project]: http://www.anti-abuse.org/multi-rbl-check/ +[tls]: https://en.wikipedia.org/wiki/Transport_Layer_Security +[ubuntu]: https://ubuntu.com +[unattended upgrades]: https://wiki.debian.org/UnattendedUpgrades diff --git a/docs/index.md b/docs/index.md index d31c493..57d1a08 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,3 @@ - # chasquid [chasquid](https://blitiri.com.ar/p/chasquid) is an SMTP (email) server with a @@ -9,16 +8,12 @@ This is the documentation index. Please see the [project page](https://blitiri.com.ar/p/chasquid) for more details. - ## Contact -If you have any questions, comments or patches please send them to the [mailing -list](https://groups.google.com/forum/#!forum/chasquid), +If you have any questions, comments or patches please send them to the +[mailing list](https://groups.google.com/forum/#!forum/chasquid), chasquid@googlegroups.com. To subscribe, send an email to chasquid+subscribe@googlegroups.com. -You can also reach out via IRC, `#chasquid` on -[OFTC](https://oftc.net/). - - +You can also reach out via IRC, `#chasquid` on [OFTC](https://oftc.net/). diff --git a/docs/install.md b/docs/install.md index 6517c59..dd3f2bd 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,4 +1,3 @@ - # Installing and configuring [chasquid](https://blitiri.com.ar/p/chasquid) ## Installation @@ -17,9 +16,9 @@ sudo apt install chasquid If you're using Arch, there is a [chasquid AUR package](https://aur.archlinux.org/packages/chasquid/) you can -use. See the [official Arch -documentation](https://wiki.archlinux.org/index.php/Arch_User_Repository) for -how to install it. If you use the [pacaur](https://github.com/E5ten/pacaur) +use. See the +[official Arch documentation](https://wiki.archlinux.org/index.php/Arch_User_Repository) +for how to install it. If you use the [pacaur](https://github.com/E5ten/pacaur) [helper](https://wiki.archlinux.org/index.php/AUR_helpers), you can just run: ```shell @@ -29,7 +28,6 @@ pacaur -S chasquid [Binary packages](https://maddy.email/archlinux/README.txt) are also available, courtesy of [foxcpp](https://github.com/foxcpp/). - ### Alpine If you're using @@ -43,7 +41,6 @@ apk add chasquid Thanks to [raspbeguy](https://github.com/raspbeguy) for maintaining the Alpine package. - ### From source To get, build and install from source, you will need a working @@ -63,7 +60,6 @@ sudo make install-binaries sudo make install-config-skeleton ``` - ## Configuration The configuration is in `/etc/chasquid/` by default, and has the following @@ -88,28 +84,24 @@ structure: ### Certificates The certs/ directory layout matches the one from -[certbot](https://certbot.eff.org/), -[letsencrypt](https://letsencrypt.org)'s -default client, to make it easier to integrate. -The layout from the [NixOS ACME module](https://wiki.nixos.org/wiki/ACME) -(`/var/lib/acme`) is also supported. +[certbot](https://certbot.eff.org/), [letsencrypt](https://letsencrypt.org)'s +default client, to make it easier to integrate. The layout from the +[NixOS ACME module](https://wiki.nixos.org/wiki/ACME) (`/var/lib/acme`) is also +supported. A convenient way to set this up is: -1) Obtain TLS certificates using `certbot` as needed. -2) Symlink chasquid's `certs/` to `/etc/letsencrypt/live`:\ +1. Obtain TLS certificates using `certbot` as needed. +2. Symlink chasquid's `certs/` to `/etc/letsencrypt/live`:\ `sudo ln -s /etc/letsencrypt/live/ /etc/chasquid/certs` -3) Give chasquid permissions to read the certificates:\ +3. Give chasquid permissions to read the certificates:\ `sudo setfacl -R -m u:chasquid:rX /etc/letsencrypt/{live,archive}` -4) Set up [automatic renewal] to restart chasquid when certificates are +4. Set up [automatic renewal] to restart chasquid when certificates are renewed. Please see the [how-to guide](howto.md#tls-certificate) for more detailed examples. -[automatic renewal]: https://eff-certbot.readthedocs.io/en/stable/using.html#setting-up-automated-renewal - - ### Adding users You can add users with: @@ -120,24 +112,21 @@ chasquid-util user-add user@domain This will also create the corresponding domain directory if it doesn't exist. - ### Checking your configuration Run `chasquid-util print-config` to parse your configuration and display the resulting values. - ### Checking your setup Run `smtp-check yourdomain.com`, it will check: -* MX DNS records. -* SPF DNS records (will just warn if not present). -* TLS certificates. - -It needs to access port 25, which is often blocked by ISPs, so it's likely -that you need to run it from your server. +- MX DNS records. +- SPF DNS records (will just warn if not present). +- TLS certificates. +It needs to access port 25, which is often blocked by ISPs, so it's likely that +you need to run it from your server. ### Greylisting, anti-spam and anti-virus @@ -146,12 +135,11 @@ greylisting, and run anti-spam and anti-virus filters. The hook should be at `/etc/chasquid/hooks/post-data`. - The one installed by default is a bash script supporting: -* greylisting using greylistd. -* anti-spam using spamassassin or rspamd. -* anti-virus using clamav. +- greylisting using greylistd. +- anti-spam using spamassassin or rspamd. +- anti-virus using clamav. To use them, they just need to be available in your system. @@ -163,6 +151,6 @@ usermod -a -G greylist mail ``` Note that the default hook may not work in all cases, it is provided as a -practical example but you should adjust it to your particular system if -needed. +practical example but you should adjust it to your particular system if needed. +[automatic renewal]: https://eff-certbot.readthedocs.io/en/stable/using.html#setting-up-automated-renewal diff --git a/docs/knownissues.md b/docs/knownissues.md index 4544837..b97f2e7 100644 --- a/docs/knownissues.md +++ b/docs/knownissues.md @@ -1,14 +1,12 @@ - # Known issues This file contains a list of the most common known issues, and the release -range affected. It can be useful for people running older versions, to -identify problems (and workarounds) faster. +range affected. It can be useful for people running older versions, to identify +problems (and workarounds) faster. Entries are eventually be purged once their affected versions become uncommon, to prevent confusion. - ## Dovecot auth occasionally not functional after a reboot (0.04 to 1.6) After a reboot, if chasquid starts *before* dovecot, it's possible that @@ -30,7 +28,6 @@ After=dovecot.service The issue is fixed in 1.7. - ## `dkimsign` causes parsing errors in post-data hook (0.07 to 1.5) The default post-data hook in versions 0.07 to 1.5 has a bug where if the diff --git a/docs/man/index.md b/docs/man/index.md index c93da4f..92a0c8b 100644 --- a/docs/man/index.md +++ b/docs/man/index.md @@ -2,8 +2,8 @@ From the latest Debian package: -- [chasquid(1)](https://manpages.debian.org/unstable/chasquid/chasquid.1): - the main binary. +- [chasquid(1)](https://manpages.debian.org/unstable/chasquid/chasquid.1): the + main binary. - [chasquid.conf(5)](https://manpages.debian.org/unstable/chasquid/chasquid.conf.5): the configuration file and structure. - [chasquid-util(1)](https://manpages.debian.org/unstable/chasquid/chasquid-util.1): @@ -17,9 +17,7 @@ From the current branch (more likely to change, but useful when doing development or running experimental versions): - [chasquid(1)](chasquid.1.md): the main binary. -- [chasquid.conf(5)](chasquid.conf.5.md): the configuration file and - structure. +- [chasquid.conf(5)](chasquid.conf.5.md): the configuration file and structure. - [chasquid-util(1)](chasquid-util.1.md): the command-line utility helper. - [mda-lmtp(1)](mda-lmtp.1.md): helper to integrate with LMTP delivery. - [smtp-check(1)](smtp-check.1.md): SMTP setup checker. - diff --git a/docs/monitoring.md b/docs/monitoring.md index 41e15b3..683317f 100644 --- a/docs/monitoring.md +++ b/docs/monitoring.md @@ -1,13 +1,12 @@ - # Monitoring -chasquid includes an HTTP server for monitoring purposes, which for security -it is not enabled by default. +chasquid includes an HTTP server for monitoring purposes, which for security it +is not enabled by default. You can use the `monitoring_address` configuration option to enable it. -Then just browse the address and human-friendly links to various monitoring -and debugging tools should appear. +Then just browse the address and human-friendly links to various monitoring and +debugging tools should appear. These include: @@ -19,7 +18,6 @@ These include: - [Exported variables](#variables) for whitebox monitoring. - Profiling endpoints, for use with `go tool pprof` or similar tools. - ## Variables chasquid exports some variables for monitoring, via the standard @@ -30,8 +28,7 @@ whitebox monitoring. They're accessible on the monitoring HTTP server, at `/debug/vars` (default endpoint for expvars) and `/metrics` (common endpoint for openmetrics). -<a name="prometheus"></a> -The `/metrics` endpoint is also compatible with +<a name="prometheus"></a> The `/metrics` endpoint is also compatible with [Prometheus](https://prometheus.io/). *Note these are still subject to change, although breaking changes will be @@ -40,86 +37,86 @@ avoided whenever possible, and will be noted in the List of exported variables: -- **chasquid/aliases/hookResults** (hook result -> counter) +- **chasquid/aliases/hookResults** (hook result -> counter)\ count of aliases hook results, by hook and result. -- **chasquid/queue/deliverAttempts** (recipient type -> counter) +- **chasquid/queue/deliverAttempts** (recipient type -> counter)\ attempts to deliver mail, by recipient type (pipe/local email/remote email). -- **chasquid/queue/dsnQueued** (counter) +- **chasquid/queue/dsnQueued** (counter)\ count of DSNs that we generated (queued). -- **chasquid/queue/itemsWritten** (counter) +- **chasquid/queue/itemsWritten** (counter)\ count of items the queue wrote to disk. -- **chasquid/queue/putCount** (counter) +- **chasquid/queue/putCount** (counter)\ number of envelopes put in the queue. -- **chasquid/smtpIn/acceptErrors** (counter) +- **chasquid/smtpIn/acceptErrors** (counter)\ count of errors accepting incoming connections. -- **chasquid/smtpIn/activeConnections** (gauge) +- **chasquid/smtpIn/activeConnections** (gauge)\ number of active incoming connections. -- **chasquid/smtpIn/commandCount** (map of command -> count) +- **chasquid/smtpIn/commandCount** (map of command -> count)\ count of SMTP commands received, by command. Note that for unknown commands we use `unknown<COMMAND>`. -- **chasquid/smtpIn/connectionsOverLimit** (counter) +- **chasquid/smtpIn/connectionsOverLimit** (counter)\ count of incoming connections closed due to the connection limit, which is half of the file descriptor limit. -- **chasquid/smtpIn/dkimSignErrors** (counter) +- **chasquid/smtpIn/dkimSignErrors** (counter)\ count of DKIM sign errors -- **chasquid/smtpIn/dkimSigned** (counter) +- **chasquid/smtpIn/dkimSigned** (counter)\ count of successful DKIM signs -- **chasquid/smtpIn/dkimVerifyErrors** (counter) +- **chasquid/smtpIn/dkimVerifyErrors** (counter)\ count of DKIM verification errors -- **chasquid/smtpIn/dkimVerifyFound** (counter) +- **chasquid/smtpIn/dkimVerifyFound** (counter)\ count of messages with at least one DKIM signature -- **chasquid/smtpIn/dkimVerifyNotFound** (counter) +- **chasquid/smtpIn/dkimVerifyNotFound** (counter)\ count of messages with no DKIM signatures -- **chasquid/smtpIn/dkimVerifyValid** (counter) +- **chasquid/smtpIn/dkimVerifyValid** (counter)\ count of messages with at least one valid DKIM signature -- **chasquid/smtpIn/hookResults** (result -> counter) +- **chasquid/smtpIn/hookResults** (result -> counter)\ count of hook invocations, by result. -- **chasquid/smtpIn/loopsDetected** (counter) +- **chasquid/smtpIn/loopsDetected** (counter)\ count of email loops detected. -- **chasquid/smtpIn/responseCodeCount** (code -> counter) +- **chasquid/smtpIn/responseCodeCount** (code -> counter)\ count of response codes returned to incoming SMTP connections, by result code. -- **chasquid/smtpIn/securityLevelChecks** (result -> counter) +- **chasquid/smtpIn/securityLevelChecks** (result -> counter)\ count of security level checks on incoming connections, by result. -- **chasquid/smtpIn/spfResultCount** (result -> counter) +- **chasquid/smtpIn/spfResultCount** (result -> counter)\ count of SPF checks, by result. -- **chasquid/smtpIn/tlsCount** (tls status -> counter) +- **chasquid/smtpIn/tlsCount** (tls status -> counter)\ count of TLS statuses (plain/tls) for incoming SMTP connections. -- **chasquid/smtpIn/wrongProtoCount** (command -> counter) +- **chasquid/smtpIn/wrongProtoCount** (command -> counter)\ count of commands for other protocols (e.g. HTTP commands). -- **chasquid/smtpOut/securityLevelChecks** (result -> counter) +- **chasquid/smtpOut/securityLevelChecks** (result -> counter)\ count of security level checks on outgoing connections, by result. -- **chasquid/smtpOut/sts/mode** (mode -> counter) +- **chasquid/smtpOut/sts/mode** (mode -> counter)\ count of STS checks on outgoing connections, by mode (enforce/testing). -- **chasquid/smtpOut/sts/security** (result -> counter) +- **chasquid/smtpOut/sts/security** (result -> counter)\ count of STS security checks on outgoing connections, by result (pass/fail). -- **chasquid/smtpOut/tlsCount** (status -> counter) +- **chasquid/smtpOut/tlsCount** (status -> counter)\ count of TLS status (insecure TLS/secure TLS/plain) on outgoing connections. -- **chasquid/sourceDateStr** (string) +- **chasquid/sourceDateStr** (string)\ timestamp when the binary was built, in human readable format. -- **chasquid/sourceDateTimestamp** (int) +- **chasquid/sourceDateTimestamp** (int)\ timestamp when the binary was built, in seconds since epoch. -- **chasquid/sts/cache/expired** (counter) +- **chasquid/sts/cache/expired** (counter)\ count of expired entries in the STS cache. -- **chasquid/sts/cache/failedFetch** (counter) +- **chasquid/sts/cache/failedFetch** (counter)\ count of failed fetches in the STS cache. -- **chasquid/sts/cache/fetches** (counter) +- **chasquid/sts/cache/fetches** (counter)\ count of total fetches in the STS cache. -- **chasquid/sts/cache/hits** (counter) +- **chasquid/sts/cache/hits** (counter)\ count of hits in the STS cache. -- **chasquid/sts/cache/invalid** (counter) +- **chasquid/sts/cache/invalid** (counter)\ count of invalid policies in the STS cache. -- **chasquid/sts/cache/ioErrors** (counter) +- **chasquid/sts/cache/ioErrors** (counter)\ count of I/O errors when maintaining the STS cache. -- **chasquid/sts/cache/marshalErrors** (counter) +- **chasquid/sts/cache/marshalErrors** (counter)\ count of marshaling errors when maintaining the STS cache. -- **chasquid/sts/cache/refreshCycles** (counter) +- **chasquid/sts/cache/refreshCycles** (counter)\ count of STS cache refresh cycles. -- **chasquid/sts/cache/refreshErrors** (counter) +- **chasquid/sts/cache/refreshErrors** (counter)\ count of STS cache refresh errors. -- **chasquid/sts/cache/refreshes** (counter) +- **chasquid/sts/cache/refreshes** (counter)\ count of STS cache refreshes. -- **chasquid/sts/cache/unmarshalErrors** (counter) +- **chasquid/sts/cache/unmarshalErrors** (counter)\ count of unmarshaling errors in the STS cache. -- **chasquid/version** (string) +- **chasquid/version** (string)\ version string. diff --git a/docs/relnotes.md b/docs/relnotes.md index e4365df..96cd34e 100644 --- a/docs/relnotes.md +++ b/docs/relnotes.md @@ -1,10 +1,8 @@ - # Release notes This file contains notes for each release, summarizing changes and explicitly noting backward-incompatible changes or known security issues. - ## 1.18.0 (2026-06-07) - Conform to the stricter OpenMetrics format in the monitoring exporter, which @@ -12,7 +10,6 @@ noting backward-incompatible changes or known security issues. - Fix extra newlines in the authentication log entries. - Docker image improvements, including updating the Dovecot config to 2.4. - ## 1.17.0 (2025-11-30) - Fix `Received` header format when including the IP address. @@ -20,7 +17,6 @@ noting backward-incompatible changes or known security issues. - Docker image improvements and bug fixes. - Minor fixes for systems without /bin. - ## 1.16.0 (2025-06-08) - Minor bug fixes to `chasquid-util`. @@ -31,7 +27,6 @@ noting backward-incompatible changes or known security issues. - Make the max queue size and give up time [configurable](man/chasquid.conf.5.md) *(experimental)*. - ## 1.15.0 (2025-01-17) - Exit if there's an error reading users/aliases files on startup. @@ -40,15 +35,14 @@ noting backward-incompatible changes or known security issues. ### 1.15.1 (2025-03-30) -Implement a workaround for a Microsoft bug in TLS session ticket handling, -that is causing deliverability issues, and they are being too slow at fixing. +Implement a workaround for a Microsoft bug in TLS session ticket handling, that +is causing deliverability issues, and they are being too slow at fixing. See this [chasquid issue](https://github.com/albertito/chasquid/issues/64), this [Go issue](https://github.com/golang/go/issues/70232) and this [Postfix thread](https://www.mail-archive.com/postfix-users@postfix.org/msg104308.html) for more details. - ## 1.14.0 (2024-04-21) - Add built-in [DKIM](dkim.md) signing and verification. @@ -59,17 +53,16 @@ for more details. [SemVer](https://semver.org/)-compatible, to help integration with other software that expects it (e.g. [pkg.go.dev](https://pkg.go.dev/)). - ## 1.13 (2023-12-24) Security fixes: -- Strict CRLF enforcement in DATA contents, to prevent [SMTP smuggling - attacks](https://www.postfix.org/smtp-smuggling.html) +- Strict CRLF enforcement in DATA contents, to prevent + [SMTP smuggling attacks](https://www.postfix.org/smtp-smuggling.html) ([CVE-2023-52354](https://nvd.nist.gov/vuln/detail/CVE-2023-52354)). \ [RFC5322](https://www.rfc-editor.org/rfc/rfc5322#section-2.3) and - [RFC5321](https://www.rfc-editor.org/rfc/rfc5321#section-2.3.8) say - that the only valid newline terminator in SMTP is CRLF. \ + [RFC5321](https://www.rfc-editor.org/rfc/rfc5321#section-2.3.8) say that the + only valid newline terminator in SMTP is CRLF. \ When an invalid newline terminator is found in an incoming message, the connection is now aborted immediately (previous releases also accepted LF-terminated lines). \ @@ -84,11 +77,10 @@ Other changes: entry in the config. - `docker/add-user.sh`: Support getting email and password from env variables. - ## 1.12 (2023-10-07) -- Support [aliases with drop characters and - suffix separators](aliases.md#drop-characters-and-suffix-separators). +- Support + [aliases with drop characters and suffix separators](aliases.md#drop-characters-and-suffix-separators). - Improved delivery on some low-level TLS negotiation errors. - `smtp-check`: Add flag to specify local name. - `chasquid-util`: `aliases-resolve` and `domaininfo-remove` subcommands now @@ -98,7 +90,6 @@ Other changes: command that was added a while ago, and there is no need for it anymore. - Handle symlinks under the `certs/` directory. - ## 1.11 (2023-02-19) - New tracing library for improved observability. @@ -110,14 +101,12 @@ Backport the security fixes from 1.13 (*Strict CRLF enforcement in DATA contents*, fixes [CVE-2023-52354](https://nvd.nist.gov/vuln/detail/CVE-2023-52354)). - ## 1.10 (2022-09-01) - Support [catch-all aliases](aliases.md#catch-all). - Fix bug in Docker image with user-provided certificates. - Miscellaneous test improvements. - ## 1.9 (2022-03-05) - Improve certificate validation logic in the SMTP courier. @@ -125,11 +114,10 @@ contents*, fixes - Support `""` values for `drop_characters` and `suffix_separators` in the configuration file. - ## 1.8 (2021-07-30) -- Stricter error checking to help prevent cross-protocol attacks - (like [ALPACA](https://alpaca-attack.com/)). +- Stricter error checking to help prevent cross-protocol attacks (like + [ALPACA](https://alpaca-attack.com/)). - Allow authenticating users without an `@domain` part. - Add integration for [chasquid-rspamd](https://github.com/Thor77/chasquid-rspamd) and @@ -138,18 +126,16 @@ contents*, fixes - Use `application/openmetrics-text` as content type in the openmetrics exporter. - ## 1.7 (2021-05-31) -- chasquid-util no longer depends on the unmaintained docopt-go. - If you relied on undocumented parsing behaviour before, your invocations may - need adjustment. In particular, `--a b` is no longer supported, and `--a=b` - must be used instead. +- chasquid-util no longer depends on the unmaintained docopt-go. If you relied + on undocumented parsing behaviour before, your invocations may need + adjustment. In particular, `--a b` is no longer supported, and `--a=b` must + be used instead. - Improve handling of errors when talking to Dovecot for authentication. - Fix handling of `hostname` option in the Docker image. - Miscellaneous documentation and test improvements. - ## 1.6 (2020-11-22) - Pass the EHLO domain to the post-data hook. @@ -157,7 +143,6 @@ contents*, fixes - Implement HAProxy protocol support (experimental). - Documentation updates. - ## 1.5 (2020-09-12) - Add OpenMetrics exporter (compatible with Prometheus). @@ -166,14 +151,12 @@ contents*, fixes - Fix rspamd greylist action handling in the default hook. - Miscellaneous monitoring server improvements. - ## 1.4 (2020-05-22) - Use the configured hostname in outgoing SMTP HELO/EHLO. - Allow config overrides from the command line. - Miscellaneous test improvements and code cleanups. - ## 1.3 (2020-04-12) - Improved handling of DNS temporary errors. @@ -181,7 +164,6 @@ contents*, fixes Arch installation instructions). - Miscellaneous test improvements and cleanups. - ## 1.2 (2019-12-06) Security fixes: @@ -197,7 +179,6 @@ Release notes: - New documentation sections (monitoring, release notes). - Many miscellaneous test improvements. - ## 1.1 (2019-10-26) - Added hooks for aliases resolution. @@ -207,7 +188,6 @@ Release notes: - Documentation and test improvements. - Minor bug fixes. - ## 1.0 (2019-07-15) No backwards-incompatible changes. No more are expected within this major @@ -217,7 +197,6 @@ version. - Make DSN tidier, especially in handling multi-line errors. - Miscellaneous test improvements. - ## 0.07 (2019-01-19) No backwards-incompatible changes. @@ -227,14 +206,12 @@ No backwards-incompatible changes. - Miscellaneous test improvements. - DKIM integration examples and test. - ## 0.06 (2018-07-22) No backwards-incompatible changes. - New MTA-STS (Strict Transport Security) checking. - ## 0.05 (2018-06-05) No backwards-incompatible changes. @@ -244,7 +221,6 @@ No backwards-incompatible changes. - Periodic reload of domaininfo, support removing entries manually. - Dovecot auth support no longer considered experimental. - ## 0.04 (2018-02-10) No backwards-incompatible changes. @@ -252,18 +228,16 @@ No backwards-incompatible changes. - Add Dovecot authentication support (experimental). - Miscellaneous bug fixes to mda-lmtp and tests. - ## 0.03 (2017-07-15) **Backwards-incompatible changes:** -- The default MTA binary has changed. It's now maildrop by default. - If you relied on procmail being the default, add the following to +- The default MTA binary has changed. It's now maildrop by default. If you + relied on procmail being the default, add the following to `/etc/chasquid/chasquid.conf`: `mail_delivery_agent_bin: "procmail"`. -- chasquid now listens on a third port, submission-on-TLS. - If using systemd, copy the `etc/systemd/system/chasquid-submission_tls.socket` - file to `/etc/systemd/system/`, and start it. - +- chasquid now listens on a third port, submission-on-TLS. If using systemd, + copy the `etc/systemd/system/chasquid-submission_tls.socket` file to + `/etc/systemd/system/`, and start it. Release notes: @@ -271,7 +245,6 @@ Release notes: - Change the default MDA binary to `maildrop`. - Add a very basic MDA that uses LMTP to do the mail delivery. - ## 0.02 (2017-03-03) No backwards-incompatible changes. @@ -280,7 +253,6 @@ No backwards-incompatible changes. - Fall back through the MX list on errors. - Experimental MTA-STS implementation (disabled by default). - ## 0.01 (2016-11-03) Initial release. diff --git a/docs/sec-levels.md b/docs/sec-levels.md index eade743..d9b7f5f 100644 --- a/docs/sec-levels.md +++ b/docs/sec-levels.md @@ -1,4 +1,3 @@ - # Security level checks chasquid tracks per-domain TLS support, and uses it to prevent connection @@ -11,28 +10,26 @@ servers. This is very different from other MTAs, and has some tradeoffs. - ## Outgoing connections An outgoing connection has one of 3 security levels, which are (in order): 1. Plain: connection is plain-text (the server does not support TLS). -2. TLS insecure: TLS connection established, but the certificate itself was - not valid. +2. TLS insecure: TLS connection established, but the certificate itself was not + valid. 3. TLS secure: TLS connection established, with a valid certificate. When establishing an outgoing connection, chasquid will always attempt to -negotiate up to the *TLS secure* level. After the negotiation, it will -compare which level it got, with the previously recorded value for this -domain: - -* If the connection level is lower than the recorded value, then the - connection will be dropped, and the delivery will fail (with a transient - failure). The delivery will be retried as usual (using other MXs if - available, and repeat after some delay). -* If the connection level is the same as the recorded value, then the +negotiate up to the *TLS secure* level. After the negotiation, it will compare +which level it got, with the previously recorded value for this domain: + +- If the connection level is lower than the recorded value, then the connection + will be dropped, and the delivery will fail (with a transient failure). The + delivery will be retried as usual (using other MXs if available, and repeat + after some delay). +- If the connection level is the same as the recorded value, then the connection will proceed. -* If the connection level is higher, chasquid will record this new value, and +- If the connection level is higher, chasquid will record this new value, and proceed. If there is no previously recorded value for this domain, a *plain* level is @@ -55,25 +52,24 @@ with MTA-STS and HTTPS, but it is not universally agreed upon. It's also why the "TLS insecure" state exists, instead of the connection being rejected directly. - ### Tradeoffs Almost all other MTAs do TLS negotiation but accept *all* certificates, even -self-signed or expired ones. chasquid operates differently, as described -above. +self-signed or expired ones. chasquid operates differently, as described above. The main advantage is that, *with domains where secure connections were -previously established*, chasquid will detect connection downgrading (caused -by malicious interception such as STARTTLS blocking, as well as -misconfiguration such as incorrectly configured or expired certificates), and -avoid communicating insecurely. +previously established*, chasquid will detect connection downgrading (caused by +malicious interception such as STARTTLS blocking, as well as misconfiguration +such as incorrectly configured or expired certificates), and avoid +communicating insecurely. The main disadvantage is that if a domain changes the configuration to a lower security level, chasquid will fail the delivery (returning a message to the -sender explaining why). Because there is no formal standard for TLS -certificate validation, and most MTAs will deliver email in this situation, -the domain owners might not see this as a problem and thus require [manual -intervention](#manual-override) on the chasquid side to explicitly allow it. +sender explaining why). Because there is no formal standard for TLS certificate +validation, and most MTAs will deliver email in this situation, the domain +owners might not see this as a problem and thus require +[manual intervention](#manual-override) on the chasquid side to explicitly +allow it. ### MTA-STS @@ -85,7 +81,6 @@ It is supported by chasquid, out of the box, and in practice it means that for domains that advertise MTA-STS support, the *secure* level will be enforced even if the domain was previously unknown. - ## Incoming connections Incoming connections from authenticated users are always done over TLS @@ -107,11 +102,11 @@ security levels, which are (in order): At this point, chasquid will compare the level with the previously recorded value for this domain: -* If the connection level is lower than the recorded value, then the - connection is rejected with an SMTP error. -* If the connection level is the same as the recorded value, then the +- If the connection level is lower than the recorded value, then the connection + is rejected with an SMTP error. +- If the connection level is the same as the recorded value, then the connection is allowed. -* If the connection level is higher, chasquid will record this new value, and +- If the connection level is higher, chasquid will record this new value, and the connection is allowed. If there is no previously recorded value for this domain, a *plain* level is @@ -136,14 +131,12 @@ MTAs will accept the connection anyway, domain owners might not even notice there is a problem, and might require [manual intervention](#manual-override) on the chasquid side to explicitly allow it. - - ## Accepting lower security levels {#manual-override} If a domain changes its configuration to a lower security level and is causing chasquid to fail delivery, you can use -`chasquid-util domaininfo-remove <domain>` to make the server forget about -that domain. +`chasquid-util domaininfo-remove <domain>` to make the server forget about that +domain. Then, the next time there is a connection, there is no high security expectation so it will proceed just fine, regardless of the level that was diff --git a/docs/security.md b/docs/security.md index 9da9576..41406d2 100644 --- a/docs/security.md +++ b/docs/security.md @@ -5,5 +5,4 @@ albertito@blitiri.com.ar, or do it via [GitHub's Security tab]. Thank you! - -[GitHub's Security tab]: https://github.com/albertito/chasquid/security +[github's security tab]: https://github.com/albertito/chasquid/security diff --git a/test/README.md b/test/README.md index 1dc9924..7dd4032 100644 --- a/test/README.md +++ b/test/README.md @@ -1,4 +1,3 @@ - # Testing ## Go tests @@ -6,46 +5,43 @@ All Go packages have their own test suite, which provides easy and portable tests with decent enough coverage. - ## Integration tests In the `test/` directory there is a set of end to end integration tests, written usually in a combination of bash and Python 3. -They're not expected to be portable, as that gets impractical very quickly, -but should be usable in most Linux environments. +They're not expected to be portable, as that gets impractical very quickly, but +should be usable in most Linux environments. They provide critical coverage and integration tests for real life scenarios, as well as interactions with other software (like Exim or Dovecot). - ### Dependencies The tests depend on the following things being installed on the system (listed as Debian package, for consistency): - - `msmtp` - - `util-linux` (for `/usr/bin/setsid`) +- `msmtp` +- `util-linux` (for `/usr/bin/setsid`) Some individual tests have additional dependencies, and the tests are skipped if the dependencies are not found: -- `t-02-exim` [Exim](https://www.exim.org/) interaction tests: - - `gettext-base` (for `/usr/bin/envsubst`) - - The `exim` binary available somewhere, but it doesn't have to be - installed. There's a script `get-exim4-debian.sh` to get it from the - archives. +- `t-02-exim` [Exim](https://www.exim.org/) interaction tests: `gettext-base` + (for `/usr/bin/envsubst`), and the `exim` binary available somewhere, but it + doesn't have to be installed. There's a script `get-exim4-debian.sh` to get + it from the archives. - `t-11-dovecot` [Dovecot](https://www.dovecot.org/) interaction tests: - - `dovecot` -- `t-15-driusan_dkim` DKIM integration tests: - - The `dkimsign dkimverify dkimkeygen` binaries, from - [driusan/dkim](https://github.com/driusan/dkim) (no Debian package yet). + `dovecot`. +- `t-15-driusan_dkim` DKIM integration tests: the + `dkimsign dkimverify dkimkeygen` binaries, from + [driusan/dkim](https://github.com/driusan/dkim) (no Debian package yet). - `t-18-haproxy` [HAProxy](https://www.haproxy.org/) integration tests: - - `haproxy` + `haproxy`. - `t-19-dkimpy`: [dkimpy](https://pypi.org/project/dkimpy/) integration tests: - - `python3-dkim` + `python3-dkim`. - `t-23-prometheus`: [Prometheus](https://prometheus.io/) integration tests: - - `prometheus promtool` + `prometheus promtool`. For some tests, python >= 3.5 is required; they will be skipped if it's not available. @@ -56,7 +52,6 @@ environment variable being functional, and will be skipped if it isn't. This works by default in most Linux systems, but note that the use of `systemd-resolved` can prevent it from working properly. - ## Stress tests Also in the `test/` directory there is a set of stress tests, which generate @@ -65,7 +60,6 @@ load against chasquid to measure performance and resource consumption. While they are not exhaustive, they are useful to catch regressions and track improvements on the main code paths. - ## Fuzz tests Some Go packages also have instrumentation to run fuzz testing against them, @@ -74,13 +68,11 @@ using the [Go native fuzzing support](https://go.dev/security/fuzz/). This is critical for packages that handle sensitive user input, such as authentication encoding, aliases files, or username normalization. - ## Command-line tool tests Each command-line tool has their own set of tests, see the `test.sh` file on their corresponding directories. - ## Docker The `test/Dockerfile` can be used to set up a suitable isolated environment to @@ -89,20 +81,18 @@ run the integration and stress tests. This is very useful for automated tests, or running the integration tests in constrained or non supported environments. - ## Automated tests There are two sets of automated tests which are run on every commit to upstream, and also weekly: -* [Github Actions](https://github.com/albertito/chasquid/actions), - configured in the `.github` directory, runs the Go tests, the integration - tests, checks for vulnerabilities, and finally - also builds the [public Docker images](docker.md). - -* [Cirrus CI](https://cirrus-ci.com/github/albertito/chasquid), - configured in the `.cirrus.yml` file, runs Go tests on FreeBSD. +- [Github Actions](https://github.com/albertito/chasquid/actions), configured + in the `.github` directory, runs the Go tests, the integration tests, checks + for vulnerabilities, and finally also builds the + [public Docker images](docker.md). +- [Cirrus CI](https://cirrus-ci.com/github/albertito/chasquid), configured in + the `.cirrus.yml` file, runs Go tests on FreeBSD. ## Coverage