MTA-STS: The Email Security Protocol Nobody Knows

SMTP encryption has a hole you can drive a truck through: any attacker in the path can strip STARTTLS and read the mail in cleartext. MTA-STS closes it. Then it makes you host a web page to do so, which is why almost nobody bothered.

Here is a fact about email that should bother you more than it does: when one mail server hands a message to another, the encryption is optional, and the decision to skip it can be made by an attacker rather than by either server.

The mechanism is STARTTLS. A sending server connects to a receiving server on port 25, says EHLO, and the receiver answers with a list of what it supports. If 250 STARTTLS is in that list, the sender upgrades the plaintext connection to TLS and the message goes encrypted. If it isn’t, the sender shrugs and sends in the clear, because for two decades the alternative to “encrypt if offered” was “don’t deliver the mail,” and refusing to deliver mail is a worse outcome than sending it unencrypted. So SMTP encryption became opportunistic: nice when it happens, never required.

You can probably already see the hole. The list of capabilities arrives before the TLS handshake — it has to, that’s how the upgrade is negotiated — which means it arrives unencrypted. An attacker sitting in the network path doesn’t need to break any crypto. They just delete the 250 STARTTLS line from the receiver’s response. The sender, seeing no offer of encryption, falls back to cleartext exactly as designed, and the attacker reads the whole message. This is the STARTTLS stripping attack, and it works because opportunistic TLS has no memory and no expectations. Every connection starts from “maybe we’ll encrypt,” and “maybe” is trivially downgraded to “no.”

You can’t fix this from inside the session

The reason this bug survived so long is that you can’t close it without information from outside the connection. Inside the SMTP session, the sender genuinely cannot tell the difference between a receiver that doesn’t support TLS and a receiver whose TLS offer got stripped. Both look identical: no STARTTLS in the EHLO response. To know that the missing offer is suspicious, the sender needs to have been told, in advance and through a channel the attacker doesn’t control, “this domain always supports TLS — if you don’t see it offered, something is wrong, don’t deliver.”

That’s the entire idea behind MTA-STS, standardized as RFC 8461 in September 2018. It’s a way for a domain to publish a durable promise: my mail servers support TLS with a valid certificate, so enforce it. Once a sending server has that policy, stripping STARTTLS stops working — the sender knows encryption should have been on the table, and when it isn’t, it refuses to deliver rather than falling back to cleartext. The downgrade attack turns into a delivery failure, which is loud and fixable, instead of a silent interception, which is neither.

If “publish a durable security policy that upgrades a connection and resists stripping” sounds familiar, it should. It’s HSTS, the header that tells browsers “only ever talk to me over HTTPS.” MTA-STS is HSTS for email — the same idea, aimed at SMTP instead of the web. Which makes the part everyone finds baffling all the more telling.

The part everyone finds baffling

To publish that policy, you run a web server.

A domain advertises MTA-STS with two pieces. The first is a DNS TXT record at _mta-sts.example.com that says, essentially, “I have a policy, and here’s a version ID” — v=STSv1; id=20260624T000000. That record is tiny and does nothing on its own except tell senders to go fetch the real thing. The real thing — the second piece — lives at a fixed HTTPS URL: https://mta-sts.example.com/.well-known/mta-sts.txt. The sender makes an HTTPS request to that subdomain and gets back the actual policy:

version: STSv1
mode: enforce
mx: mail.example.com
mx: *.example.com
max_age: 604800

mode is the lever. enforce means refuse delivery if TLS can’t be negotiated to a listed MX with a valid certificate. testing means try, but deliver anyway and report failures — the on-ramp, so you can find broken configs before they bounce real mail. none means you’re winding the policy down. mx lists which mail servers are legitimate (wildcards allowed), so an attacker can’t just point you at their own server. max_age is how many seconds to cache the policy, up to a year.

The choice to serve the policy over HTTPS rather than DNS is the clever, contentious core of the whole design. HTTPS gives you authentication for free — the policy is delivered over a connection validated against the Web PKI, the same certificate authorities your browser already trusts, so a network attacker can’t forge or tamper with it. No new trust infrastructure required. That’s the pragmatic win, and it’s the reason MTA-STS exists at all instead of the technically cleaner alternative.

The cleaner alternative that nobody could deploy

The technically cleaner alternative is DANE. DANE (RFC 7672 for SMTP) publishes the mail server’s certificate fingerprint directly in DNS, in a TLSA record, and the sender checks the cert against it on every connection. No web server, no trust-on-first-use gap, no dependence on commercial certificate authorities. On paper it’s strictly better than MTA-STS.

It has one requirement: DNSSEC. The TLSA record is only trustworthy if the DNS answer carrying it is cryptographically signed, because otherwise the same attacker who strips STARTTLS could just forge the TLSA record. So DANE inherits all of DNSSEC’s adoption problems — and DNSSEC, twenty-some years in, still isn’t deployed on the majority of domains, because signing a zone and managing key rollovers is genuinely hard and a broken chain takes your domain offline. DANE is gated behind a dependency most of the internet never adopted.

This is the fork in the road that explains everything about MTA-STS. The authors looked at DANE, saw a beautiful protocol stranded behind DNSSEC, and made a deliberate trade: give up the cryptographic purity, lean on the Web PKI that everyone already has, and get something deployable today without asking domains to first solve DNSSEC. Google made the same bet at scale — it announced MTA-STS support for Gmail in 2019, the first major provider to do so, and to this day supports MTA-STS but not DANE, almost certainly because betting on HTTPS reaches far more of the internet than betting on DNSSEC.

The seam in the design

The trade isn’t free, and it’s worth being honest about where the seam is. MTA-STS is trust-on-first-use. The very first time a sender contacts your domain, it has no cached policy yet, so it has to fetch one — and if an attacker can block that first HTTPS request, the sender never learns the policy exists and the protection silently doesn’t apply. The same window reopens, narrowly, every time a cached policy expires and has to be refreshed. DANE doesn’t have this gap, because the check happens in DNS on every single connection.

The RFC is candid about this and gives the right mitigation: refresh aggressively. Don’t wait for the cached policy to expire — re-fetch it in the background on a schedule, daily or weekly, so an attacker would have to block policy discovery continuously over the entire cache lifetime to keep a sender in the dark, instead of just winning one race at one moment. It narrows the seam to almost nothing. It doesn’t quite weld it shut. That’s the honest cost of choosing deployability over DNSSEC, and for most domains it’s obviously the right cost to pay, because a protocol you actually turn on beats a stronger one you never deploy.

Turn it on in testing and read the reports

There’s a companion protocol you should turn on at the same time: TLS-RPT (RFC 8460), a second TXT record that asks sending servers to mail you a daily summary of how TLS negotiation went — successes, failures, and crucially, policy-application failures. Without it you’re deploying a security control blind, with no way to know if enforce mode is quietly bouncing legitimate mail because one of your MX hosts has a certificate problem. With it, you start in testing mode, watch the reports for a couple of weeks, fix whatever they surface, and only then flip to enforce. The reports are the difference between a deployment and a guess.

So: MTA-STS is a real fix for a real, decades-old hole in email encryption, deliberately engineered to be deployable without DNSSEC, blessed by the largest mailbox provider on earth — and it’s still on a small minority of domains. Not because it’s controversial. Because closing the hole means standing up a dedicated mta-sts. subdomain with its own certificate to serve one static text file, and adding two more TXT records to a DNS zone that’s already crowded with SPF and DKIM and DMARC, for a threat — an active attacker stripping STARTTLS in the network path — that feels abstract until the day it isn’t. The protocol nobody knows isn’t unknown because it’s bad. It’s unknown because the work is boring and the danger is quiet, and that combination is how most good security dies.

Continue the conversation

← Back to Blog