Key exchange and hybrid encryption

You now have two tools. Symmetric encryption is fast, but both sides need the same secret key. Key pairs solve the delivery problem, but public-key operations are slow and can only handle small amounts of data. Real systems combine them. This chapter shows how.

Start with a surprising fact: two people can agree on a shared secret while someone listens to every word they say, and the listener still does not learn the secret. This is called key exchange, or key agreement, and the first method was published by Whitfield Diffie and Martin Hellman in 1976. It is named after them: Diffie-Hellman.

It works like this:

1. Ken and the harbour each make a fresh key pair, just for this conversation.
2. They swap their public keys in the open. Eve, the spy, copies both.
3. Ken combines his private key with the harbour’s public key. The harbour combines its private key with Ken’s public key.
4. The maths guarantees that both get the same result: a shared secret.
5. Eve has both public keys, but no private key, so she cannot compute that result.

Notice what did not happen: the secret never travelled. Nobody encrypted it and sent it. Each side worked it out on its own.

Then the fast part takes over. Both sides turn the shared secret into a session key and use ordinary symmetric encryption, such as AES, for the actual data: the web page, the video, the file. Using public-key maths to set up a key, and symmetric encryption for the bulk data, is called hybrid encryption. Almost every secure connection you make works this way.

One more idea makes it powerful. If both sides throw away their temporary private keys when the conversation ends, nobody can ever recompute that session key, not even the two of them. A spy who recorded the conversation and later steals a server’s long-term key still cannot read it. This is called forward secrecy.

There is one thing key exchange cannot do on its own: tell you who is on the other end. Ken agreed a secret with someone. Proving that the someone is really the harbour needs a digital signature and a way to know whose key is whose.

Swipe sideways to see the whole diagram.

How does Diffie-Hellman work with numbers?

Classic Diffie-Hellman uses arithmetic “modulo” a large prime p, with a public base g. Here it is with toy numbers, p = 23 and g = 5:

Both reach 2, because (gᵇ)ᵃ and (gᵃ)ᵇ are both gᵃᵇ. Eve sees 23, 5, 8 and 19. To get the secret she would have to find a from 8 = 5ᵃ mod 23: the “discrete logarithm problem”. With toy numbers she can try every value. With a prime of 2,048 bits or more, or with elliptic curves, nobody can.

What is ECDHE?

Modern systems do the same thing on an elliptic curve, which gives the same security with much smaller numbers. This is elliptic-curve Diffie-Hellman (ECDH). When each key pair is fresh for every session and then deleted, it is ephemeral: ECDHE.

The two curves you will see most:

• X25519 (RFC 7748, section 6.1): private keys, public keys and the shared secret are each 32 bytes.
• P-256, also called secp256r1 or prime256v1: the public value is 65 bytes (an uncompressed point), and the shared secret is the 32-byte x-coordinate of the shared point.

NIST’s key-agreement standard, SP 800-56A Rev. 3, specifies Diffie-Hellman and elliptic-curve Diffie-Hellman for approved groups and curves such as P-256.

How does TLS 1.3 use key exchange?

TLS 1.3 is defined in RFC 9846. The key exchange happens in the first two messages:

1. The client’s ClientHello lists the groups it supports and includes a fresh public key for one or more of them in the key_share extension (section 4.3.8).
2. The server picks one group and answers in its ServerHello with its own fresh public key.
3. Both compute the same shared secret (section 7.4.2).
4. The shared secret goes into the key schedule, which uses HKDF (RFC 5869) to derive separate keys for the handshake and for application data in each direction (section 7.1).
5. From then on, every record is protected with an AEAD such as AES-GCM.

TLS 1.3 made ephemeral key exchange the only public-key option. The RFC’s list of major changes says: “Static RSA and Diffie-Hellman cipher suites have been removed; all public-key based key exchange mechanisms now provide forward secrecy” (section 1.3).

So what does the server’s certificate key do? It signs. In the CertificateVerify message, the server signs a hash of the handshake so far with the private key that matches its certificate. That proves it holds the key and ties the key exchange to the certificate (section 4.5.2). The certificate key never encrypts the session key. The full sequence is in the TLS 1.3 handshake.

Swipe sideways to see the whole diagram.

What did older TLS do?

In TLS 1.2 and earlier, a server could also use “RSA key transport”: the client made the secret, encrypted it with the server’s RSA public key from the certificate, and sent it. That has no forward secrecy. Anyone who recorded the traffic and later obtained the server’s private key could decrypt every recorded session. It is why TLS 1.3 removed it, and why TLS 1.2 configurations should only offer ECDHE suites.

Why is key exchange not enough on its own?

Diffie-Hellman agrees a secret with whoever answers. An active attacker who can sit between both parties, a man-in-the-middle, can run one exchange with each side. Both sides then share a secret with the attacker instead of with each other, and the attacker can read and change everything in between. The maths is not broken; the problem is that nobody checked identities.

The defence is authentication: one or both sides sign the exchange with a long-term key, and the other side checks that the key really belongs to the expected name. In TLS that is the certificate and CertificateVerify. In SSH it is the host key. The chapter on binding identity shows why this check is the foundation of PKI.

What does “hybrid” mean in post-quantum key exchange?

You will meet “hybrid” in a second sense. A large quantum computer could solve the discrete logarithm problem, so an attacker could record today’s key exchanges and decrypt them later: “harvest now, decrypt later”. The answer is to combine a classical exchange with a post-quantum one, so the session stays safe if either holds. The best known is X25519MLKEM768, which pairs X25519 with ML-KEM-768, a post-quantum key-encapsulation mechanism. NIST’s first post-quantum standards are FIPS 203 (ML-KEM), FIPS 204 (ML-DSA) and FIPS 205 (SLH-DSA). Today, X25519MLKEM768 hybrid post-quantum key exchange is on by default in Chrome, Firefox and Apple's operating systems. See post-quantum cryptography.

The other common use of “hybrid encryption” is the one from this chapter: public-key key setup plus symmetric bulk encryption. HPKE (RFC 9180), CMS and OpenPGP all work this way for files and messages.

Watch two parties agree a secret

You can run X25519 key agreement yourself with OpenSSL. Make a key pair for each side and export the public halves:

openssl genpkey -algorithm X25519 -out alice-key.pem
openssl pkey -in alice-key.pem -pubout -out alice-pub.pem
openssl genpkey -algorithm X25519 -out bob-key.pem
openssl pkey -in bob-key.pem -pubout -out bob-pub.pem
cat alice-pub.pem

-----BEGIN PUBLIC KEY-----
MCowBQYDK2VuAyEA7I4IWWWgkrr5EJ8/x7z4PuLB5nYNNyV427ToqY4ciEg=
-----END PUBLIC KEY-----

Now each side combines its own private key with the other side’s public key (-derive with -peerkey):

openssl pkeyutl -derive -inkey alice-key.pem -peerkey bob-pub.pem | xxd -p -c 64
openssl pkeyutl -derive -inkey bob-key.pem -peerkey alice-pub.pem | xxd -p -c 64

8ebe549a8252f5843543061f8ce21af4136bb43dfff75107fc49ce6fbac8d743
8ebe549a8252f5843543061f8ce21af4136bb43dfff75107fc49ce6fbac8d743

Both lines are the same 32-byte secret, and it never existed in any file the two sides swapped. Only alice-pub.pem and bob-pub.pem would cross the network. In a real protocol this raw secret is never used directly as a key; it goes through a key derivation function such as HKDF first.

See the key exchange your server negotiates

openssl s_client shows the group used for the exchange in the Server Temp Key line. By default, OpenSSL 3.2 offered X25519:

openssl s_client -connect www.rfc-editor.org:443 -servername www.rfc-editor.org -brief < /dev/null

Protocol version: TLSv1.3
Ciphersuite: TLS_AES_256_GCM_SHA384
Signature type: ECDSA
Server Temp Key: X25519, 253 bits

Ask for a specific group with -groups to test what a server supports:

openssl s_client -connect www.rfc-editor.org:443 -servername www.rfc-editor.org -groups P-256 -brief < /dev/null

Protocol version: TLSv1.3
Ciphersuite: TLS_AES_256_GCM_SHA384
Signature type: ECDSA
Server Temp Key: ECDH, prime256v1, 256 bits

(We trimmed the connection lines from both outputs.) Newer OpenSSL versions and browsers may show a hybrid post-quantum group here instead.

Check that your TLS 1.2 settings use ECDHE

TLS 1.3 always uses ephemeral key exchange (apart from a rarely used mode that relies only on a pre-shared key). TLS 1.2 does not, so check what your configuration allows. openssl ciphers -v shows each suite’s key exchange (Kx) and authentication (Au) separately:

openssl ciphers -v -tls1_2 'ECDHE-ECDSA-AES256-GCM-SHA384:AES256-GCM-SHA384'

TLS_AES_256_GCM_SHA384         TLSv1.3 Kx=any      Au=any   Enc=AESGCM(256)            Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256   TLSv1.3 Kx=any      Au=any   Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256         TLSv1.3 Kx=any      Au=any   Enc=AESGCM(128)            Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384  TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256)            Mac=AEAD
AES256-GCM-SHA384              TLSv1.2 Kx=RSA      Au=RSA   Enc=AESGCM(256)            Mac=AEAD

Three things to read here:

• The TLS 1.3 suites say Kx=any Au=any: in TLS 1.3 the suite only names the AEAD and hash, and the key exchange is negotiated separately.
• ECDHE-ECDSA-… uses ephemeral key exchange (Kx=ECDH) and an ECDSA certificate to authenticate. This one has forward secrecy.
• AES256-GCM-SHA384 uses Kx=RSA: RSA key transport, no forward secrecy. Remove suites like this from TLS 1.2 configurations.

What breaks in practice

• Old clients that only speak static RSA. Turning off RSA key transport can break very old devices. Find them first in your logs, then plan their upgrade rather than keeping a weak default for everyone.
• Proxies and appliances in the path. Load balancers and TLS-inspecting appliances negotiate their own key exchange and may support fewer groups than the endpoints. Hybrid post-quantum key shares also make the ClientHello larger. Before you change key-exchange settings, test through every device in the path.
• Debugging encrypted traffic. Because session keys exist only in the two endpoints’ memory, a packet capture alone cannot be decrypted, even with the server’s private key. That is forward secrecy working as designed. Debug at the endpoint, with logs.

Common mistakes

• “HTTPS encrypts my data with RSA” or “with the certificate”. The data is encrypted with a symmetric cipher such as AES-GCM. The keys come from an ephemeral (EC)DHE exchange. The certificate’s key only signs, to prove the server’s identity.
• “Key exchange sends the key, just encrypted.” With Diffie-Hellman nothing secret is sent at all. Each side computes the secret. (Old RSA key transport did send it, which is why TLS 1.3 removed it.)
• “Diffie-Hellman stops man-in-the-middle attacks.” On its own it does not. It protects against listening, not against someone who answers in the other party’s place. You need authentication on top.
• “If the server’s key is stolen, all recorded traffic is exposed.” Not with ephemeral key exchange. Past sessions stay protected. The stolen key does let the thief impersonate the server, and because many clients check revocation only loosely, that risk can last until the certificate expires. Replace the key and get the certificate revoked at once.
• “Ephemeral means short-lived and therefore weaker.” Ephemeral keys are just as strong. They are thrown away after use, which is exactly what protects past sessions.

Check yourself

Check yourself

Eve records a complete ECDHE key exchange between Ken and the harbour: both public keys and everything else on the wire. Can she compute their shared secret?

1. Yes, she has both halves of the exchange
2. No: computing the secret needs one of the private keys, which never crossed the network
3. Only if she also records the encrypted data afterwards

Show answer

Answer B No: computing the secret needs one of the private keys, which never crossed the network

What happened

Eve has only the two public keys.

Why

Each side computes the secret from its own private key and the other's public key. Getting a private key back from a public key is the discrete logarithm problem, which is infeasible for real key sizes.

The rule

Key exchange agrees a secret in public without sending it. Listening is not enough to learn it.

Check yourself

A web server's certificate private key is stolen in 2027. An attacker recorded TLS 1.3 traffic to that server in 2026. What can the attacker do?

1. Decrypt all the recorded 2026 traffic with the stolen key
2. Not decrypt the recordings, because each session used fresh ephemeral keys, but impersonate the server while the certificate is still valid
3. Nothing, because TLS 1.3 certificates cannot be misused

Show answer

Answer B Not decrypt the recordings, because each session used fresh ephemeral keys, but impersonate the server while the certificate is still valid

What happened

The recorded sessions used ECDHE, whose temporary private keys were deleted long ago.

Why

In TLS 1.3 the certificate key only signs the handshake. It never encrypted any session key, so it cannot unlock old recordings. It can still sign new handshakes, and many clients check revocation only loosely, which is why you replace the key and revoke the certificate at once.

The rule

Ephemeral key exchange gives forward secrecy. A stolen long-term key threatens the future, not the past.

Check yourself

ECDHE already gives both sides a shared secret. Why does a TLS 1.3 server also sign the handshake with its certificate key?

1. To encrypt the session key for the client
2. Because ECDHE alone does not prove who is on the other end; the signature ties the exchange to the key in the server's certificate
3. It does not; the signature is optional for servers

Show answer

Answer B Because ECDHE alone does not prove who is on the other end; the signature ties the exchange to the key in the server's certificate

What happened

The CertificateVerify signature proves the server holds the private key for its certificate.

Why

Without authentication, a man-in-the-middle could run one key exchange with each side. The signature covers the handshake, including both key shares, so a substitute exchange would not verify.

The rule

Key exchange gives secrecy; signatures with a trusted key give identity. Secure connections need both.

Go deeper

• RFC 9846, the TLS 1.3 specification: key shares (section 4.3.8), the key schedule (section 7.1) and (EC)DHE (section 7.4).
• RFC 7748: X25519 and X448, with test vectors you can check against OpenSSL.
• NIST SP 800-56A Rev. 3: Diffie-Hellman and elliptic-curve key agreement schemes.
• RFC 5869: HKDF, the function that turns a shared secret into keys.
• RFC 9180: HPKE, hybrid public-key encryption for files and messages.
• OpenSSL pkeyutl documentation: the -derive and -peerkey options used above.

In the gameLevel 4: Agreeing a secret in publicThe Lighthouse, Keystone. In development

Short on time? The crash course covers this chapter and the previous one in Lesson 2: Two keys.