The most crucial protocol to harden with respect to mail infrastructure is SMTP. This protocol serves a double service:

1. Submitting mail from a Mail User Agent (e.g., Thunderbird, Outlook, Evolution, …) to a Mail Submission Agent or combined Mail Transfer Agents and Mail Submission Agents (e.g., sendmail, qmail, Exchange);
2. Relaying/Forwarding mail from one MTA to a different MTA (e.g., from alice@example.com to bob@example.org).

Historically, both of these used SMTP over port 25 (ingoing and outgoing!) between the MUA↔MSA/MTA and MTA↔MTA.

Nowadays, this connection mode is mostly reserved for the latter use (relaying server-to-server) and mail submission (client-to-server) is done using port 465 with dedicated TLS/SSL tunneling or using STARTTLS on port 587. Both methods (ideally) setup an encrypted channel between the MUA and the MSA/MTA because the login credentials for the user are exchanged over this channel.

<aside> ℹ️

STARTTLS is a method to “upgrade” an unencrypted SMTP session to a TLS-encrypted one. The MUA requests this upgrade which the server can allow or deny (e.g., if unsupported). This poses the first security risk: Even if the server supports upgrading, a Man-in-the-Middle (MitM) attacker could intercept the upgrade request and deny it, thus forcing the client to use the unencrypted channel for the transmission of the password. The only way to prevent this is to configure the clients to not transmit the password if the STARTTLS handshake failed; or to use direct TLS/SSL tunneling. While the server may not accept logging in without a successful STARTTLS, for a MitM-scenario this is not a sufficient hardening since the credentials would have already been leaked to the adversary.

</aside>

<aside> 📌

If possible, the submission server should only support TLS/SSL through a dedicated tunnel and all clients migrated away from STARTTLS. Additionally, submission over port 25 should be disabled, and this port only used for relaying (see below).

</aside>

In contrast, MTA↔MTA communication does not verify user credentials and remains on port 25. This channel is unencrypted and there does not exist an equivalent to the dedicated TLS/SSL tunnel as for submission. However, if both servers support it, they may upgrade to an encrypted channel using STARTTLS on port 25 too.

<aside> 📌

Any MTA should offer STARTTLS for mail relaying on port 25.

</aside>

<aside> ℹ️

These are the next two security risks associated with SMTP: First, since MTA↔MTA communication is not authorized through any user authentication, a rogue MTA can pretend to have an email from [email protected] which they need to relay to your company. In principle, the inbound MTA would accept this mail since it came from another MTA on port 25. Secondly, if either of the servers does not support STARTTLS — or they do support it but we have a MitM-attack just like above — the transfer/relay of any emails are unencrypted and thus readable by any intermediary.

</aside>

<aside> 📌

To announce the support of STARTTLS for mail relaying MTA-STS or DANE can be used. Both must be combined with DNSSEC to secure against DNS Spoofing. In both cases, the sending MTA can use a different, secured (through DNSSEC), channel to lookup whether the server supports STARTTLS. If the server insists on not wanting to establish a STARTTLS connection despite the MTA-STS/DANE provided information stating otherwise, a MitM-attack is likely underway.

</aside>

<aside> 📌

The MTA should also check the MTA-STS/DANE records of other MTAs before relaying messages. The MTA may even be configured to never send mail over unencrypted channels and bounce them instead (most modern mail servers support STARTTLS).

</aside>

In order to secure mail infrastructure, Internet Service Providers (ISPs) typically disallow opening port 25 on regular contracts to prevent abuse.

<aside> 📌

Any MTA should be configured to not allow email relaying from clients on a port different than 25.

</aside>

Whilst ISPs are quite restrictive handing out port 25 nowadays, this is not a sufficient security measure. By default, this would still allow anyone with such access to send mail on the behalf of anyone else through your server; this will, if the recipient’s mail server is well-configured with SPF/DKIM, see below, be bounced and your mail server likely reported and put on a denylist/spamlist in consequence.

<aside> 📌

In order to not become a so-called Open Relay, MTAs should never accept mail from non-local IP addresses to other non-local IP addresses.

</aside>

Finally, to replace the missing authentication & authorization of the user sending the mail when relaying it — and in doing so defend against other Open Relays and other badly configured mail servers —, the receiving MTA must have some way to verify that the sender is actually allowed to send mails on behalf of the sending domain. This is done through additional DNS records, which

1. announce what MTAs are allowed to send a mail from a specified domain in the mails envelope From: header (SPF), and
2. establish a signature scheme for mail content to be verifiably from a reputable server with a cryptographic public key associated with that domain (DKIM).

Thus, SPF works on the MTA level to drop mails if they’re received from a server that’s not authorized to send from the given domain. DKIM allows the MUA to verify a single mail’s sender domain.