TLS handshake failed when sending — how to fix it

A TLS handshake failure means your server and the receiver couldn't agree on a secure connection — so a receiver that expects TLS defers or holds the mail. It looks alarming, but it's a configuration problem with a short list of causes — and because the handshake is a defined sequence of steps, the failure almost always points to exactly which step broke and on whose side.

How the handshake works, and where it breaks

On port 25 the connection starts in plaintext. The receiving server advertises STARTTLS, your server sees it and issues the STARTTLS command to upgrade the connection, and then the two negotiate: they agree a TLS version, agree a cipher they both support, and the receiver presents its certificate, which your server may validate. Only when all of that succeeds does encrypted delivery proceed.

A handshake failure is one of those steps not completing, and each maps to a cause. No STARTTLS offered (or it was stripped) means there’s nothing to upgrade to. No shared version means a protocol mismatch — typically you offering only old TLS while the receiver requires 1.2 or newer. No shared cipher means a cipher mismatch. A certificate that’s expired, incomplete or doesn’t match the hostname fails validation. Knowing which step broke turns “TLS handshake failed” from a vague error into a specific, fixable one.

Make sure STARTTLS is offered

PowerMTA attempts opportunistic STARTTLS by default. If TLS was disabled, or your server never advertises STARTTLS, a receiver that requires it has nothing to upgrade to and defers the message. Confirm STARTTLS is enabled and offered on the outbound path before looking further.

Offer modern protocols and ciphers

Strict receivers reject obsolete TLS. If your configuration still offers a deprecated protocol version, or a cipher set the receiver won't accept, the handshake fails outright. Offer TLS 1.2 and 1.3 with current ciphers and remove the old ones — this resolves a large share of handshake failures.

On the sending side specifically, the usual root cause is an outdated TLS stack: an old OpenSSL that doesn’t speak TLS 1.3, or a hand-edited cipher list that’s become empty or full of ciphers modern receivers no longer accept. Update OpenSSL, set a sane minimum protocol of TLS 1.2, and use a current cipher string rather than a hand-curated one that rots over time. TLS 1.3 is the 2026 standard — a faster one-round-trip handshake with stronger ciphers and mandatory forward secrecy — so the goal is to offer 1.2 and 1.3 and let the receiver pick, while 1.0 and 1.1 stay disabled. In PowerMTA or KumoMTA this is a few lines of TLS configuration; the same principle applies to either engine.

Validate the certificate chain

For receivers that validate your certificate — and for any domain enforcing MTA-STS — an expired, incomplete or hostname-mismatched certificate breaks the handshake. Make sure the full chain is present, current, and matches your sending hostname.

The most common certificate fault is an incomplete chain — the leaf certificate is present but the intermediate is missing, so a validating receiver can’t build a path to a trusted root. Install the full chain (leaf plus intermediates), rather than the leaf certificate alone. The next most common is a hostname mismatch: the certificate’s Subject Alternative Name doesn’t include the hostname the connection used, which fails validation for strict receivers and for MTA-STS. A self-signed or expired certificate fails the same way. Verify all of this with the openssl x509 inspection above — it shows the dates, the issuer chain and the SAN list in one go — and reissue or complete the certificate so it’s current, fully chained, and covers the right hostname.

Diagnose with openssl s_client

You don’t have to guess which step failed — reproduce the handshake by hand. From the sending server, run an openssl s_client STARTTLS probe against the destination MX and read what comes back: the negotiated protocol and cipher, the certificate it presented, and whether verification passed.

# negotiate STARTTLS and show the result
printf 'QUIT\r\n' | openssl s_client -starttls smtp \
  -connect aspmx.l.google.com:25 -servername aspmx.l.google.com

# inspect the presented certificate (dates, issuer, SAN)
printf 'QUIT\r\n' | openssl s_client -starttls smtp \
  -connect aspmx.l.google.com:25 -servername aspmx.l.google.com 2>/dev/null \
  | openssl x509 -noout -subject -issuer -dates -ext subjectAltName

Look for three things. The Protocol and Cipher lines tell you what was actually agreed — if the connection won’t complete, a version or cipher mismatch is likely. The certificate notAfter date shows expiry. And the Subject Alternative Name must include the MX hostname — if it doesn’t, that’s a host-mismatch that breaks validation for any receiver (or policy) that checks it. A verify return code other than 0 (ok) names the validation problem directly.

Is it your side or theirs?

The single most useful split: does the handshake fail to every destination, or just one? If TLS fails everywhere, the problem is your own outbound stack — an outdated OpenSSL, a configuration offering only obsolete protocols, or an empty or broken cipher list — and the fix is on your server. If it fails to a single recipient while others succeed, the problem is at that destination: their certificate is expired or mis-issued, their MX presents a cert whose SAN doesn’t match, or they require something your server doesn’t offer.

The openssl probe settles it immediately — run it against the failing destination and against a known-good one like Google’s MX. If the good one negotiates cleanly and the failing one doesn’t, you’ve localised the fault to the receiver; if even the good one fails, your stack is the problem. That one comparison saves you from reconfiguring a perfectly healthy server to chase a fault that lives elsewhere.

Opportunistic vs enforced: MTA-STS and DANE

By default STARTTLS is opportunistic: if the receiver offers it, your server encrypts; if not, it falls back to plaintext. That convenience is also a weakness — a network attacker can strip the STARTTLS advertisement and force the connection back to clear text (a downgrade attack). Two standards close that gap by letting a receiving domain declare “TLS is mandatory for my mail”: MTA-STS (published over HTTPS) and DANE (anchored in DNSSEC). When a destination publishes one of these in enforce mode, your server must establish valid TLS or the message isn’t delivered.

This is why a TLS handshake problem turns from a soft issue into a hard delivery blocker against TLS-enforcing domains: there’s no plaintext fallback to save you. The fix is never to route around the policy — it’s to make your TLS negotiate cleanly so validation passes. If you are deploying MTA-STS on your own receiving domain, do it in stages: start in testing mode, which behaves as if TLS is mandatory but still delivers on failure and sends you a TLS-RPT report explaining why, and only move to enforce once those reports are clean. Jumping straight to enforce is how people discover a forgotten backup MX with an expired certificate — by bouncing real mail.

TLS-RPT is the feedback loop worth wiring up either way: it reports which sources failed to deliver to TLS-enforcing domains and why, so a certificate or protocol problem surfaces as a report rather than as silent lost mail. Honour recipients’ MTA-STS and DANE policies, keep your own TLS current, and the enforced-TLS world becomes something you pass cleanly rather than fight.

A fix checklist

Once you’ve localised the fault, the repair is usually one of a short list. Work them in this order:

1. Confirm STARTTLS is enabled and offered on your outbound path — if it was disabled, nothing else matters.
2. Update OpenSSL and set a minimum of TLS 1.2, offering 1.2 and 1.3 and disabling 1.0/1.1.
3. Use a current cipher string rather than a stale hand-edited list that may be empty or full of rejected ciphers.
4. Install the full certificate chain (leaf plus intermediates), current and not expired.
5. Match the certificate to the hostname so the SAN covers the MX hostname, for strict and MTA-STS receivers.
6. Re-probe with openssl to the affected destination and confirm a clean negotiation and verify return code: 0.

If the probe still fails after your side is clean — modern protocols, current ciphers, a valid full-chain certificate — and only one destination is affected, the fault is theirs to fix, and the right move is to flag it rather than weaken your own configuration to accommodate a broken receiver.

When it’s a middlebox, not the endpoints

Occasionally both endpoints are healthy and a device in between is breaking TLS. A firewall, an SMTP proxy, a DLP appliance or a “SMTP fixup” feature on a router can interfere with the STARTTLS exchange — some older firewall SMTP-inspection features are notorious for mangling the STARTTLS command or the negotiation that follows. The symptom is a handshake that fails in ways the endpoints’ own configurations don’t explain.

Suspect this when your openssl probe behaves differently from a probe run from outside your network, or when failures cluster around a particular network path rather than a particular receiver. The fix is to disable SMTP/STARTTLS inspection on the offending device for your mail traffic, or to route outbound mail so it doesn’t pass through it. It’s an uncommon cause, but a maddening one to chase if you’re only looking at the two mail servers.

Opportunistic vs mandatory: the levels

It helps to think of outbound TLS as a spectrum of strictness rather than on/off, because the right setting differs by route. Most MTAs expose roughly these levels:

Level	Behaviour	Use for
None / plaintext	No TLS attempted	Effectively never — modern receivers expect TLS
Opportunistic	Use TLS if offered, else plaintext	The general-purpose default for most mail
Mandatory (encrypt)	Require TLS or defer/bounce	Sensitive routes where clear text is unacceptable
Validated (MTA-STS / DANE)	Require valid TLS, honouring the receiver’s policy	Destinations that publish an enforcement policy

The sensible posture for general mail is opportunistic STARTTLS with TLS 1.0 and 1.1 disabled, escalating to validated TLS for destinations that demand it. Making it a binary choice between “old TLS allowed” and “total bounce risk” is the wrong frame — you tune strictness per route, keep the common case opportunistic, and let receivers’ published policies pull specific routes up to validated.

Why TLS matters more now

Transport encryption has gone from nice-to-have to baseline. The overwhelming majority of mail to large providers now travels over TLS, and the share that doesn’t is increasingly treated as suspect. That shift is why a handshake failure that a few years ago meant a quiet plaintext fallback now increasingly means a deferral or a hold — the fallback is being taken away on purpose, both by individual receivers and by the spread of MTA-STS and DANE.

For a legitimate sender that’s good news, because clean modern TLS is easy to provide and it’s one more signal of a well-run operation. The practical implication is simply to stop treating TLS as optional plumbing: keep the stack current, monitor for handshake failures the way you monitor bounces and complaints, and fix them promptly, because a persistent TLS problem to a major destination is now a deliverability problem, not a cosmetic one.

After the fix: confirm and keep watching

Once you’ve changed something — updated OpenSSL, adjusted the protocol or cipher set, installed a complete certificate — prove it before trusting it. Re-run the openssl s_client probe to the destination that was failing and confirm you now get a clean negotiation with verify return code: 0 (ok), then watch the previously-deferred mail actually drain from the queue. A configuration that looks right but hasn’t been re-probed isn’t verified — TLS is exactly the area where an untested assumption bites.

Then keep an eye on it, because certificates expire and receivers tighten their requirements over time. A certificate that validates today fails the day it lapses, and a protocol that’s accepted now may be refused after a receiver raises its minimum. Treat certificate renewal as a monitored task rather than a yearly surprise, and watch your logs and any TLS-RPT reports for a rising rate of handshake failures the same way you watch bounces — a TLS problem caught from a report is a quick fix, while one discovered from missing mail is a scramble.

The bottom line

A TLS handshake failure on outbound mail is a negotiation that didn’t complete, and the handshake’s defined steps tell you which one broke: no STARTTLS offered, no shared protocol, no shared cipher, or a certificate that’s expired, incomplete or hostname-mismatched. Reproduce it with an openssl s_client probe, compare a failing destination against a known-good one to tell your side from theirs, and fix accordingly — update your TLS stack and ciphers if it’s yours, flag the receiver if it’s theirs.

The one thing never to do is downgrade to plaintext to “get it through” — TLS-enforcing domains (via MTA-STS or DANE) will hold the mail regardless, and you’d be weakening security for every other recipient. Keep your outbound TLS current, honour recipients’ enforcement policies, and wire up TLS-RPT so failures surface as reports rather than silent losses. If you’d rather not hand-tune OpenSSL, ciphers and certificates, the Auto PMTA Configurator sets up modern TLS, valid certificates and MTA-STS during deploy, and our deliverability audit will pin down a stubborn handshake failure that resists the usual checks — so encrypted delivery to the strict, TLS-enforcing receivers simply works, for legitimate, opt-in senders.