> How cross-signing works, why roots expire, what the DST Root CA X3 expiry broke, how clients choose a path, and why pinning makes rotation fragile. Source: https://pkizerotohero.com/handbook/cross-signing/ · Last reviewed: 2026-10-20 Part IV: Trust and HTTPS · Chapter 14 # Two roads to the same root: cross-signing and root expiry **In one sentence:** Cross-signing gives one CA key two certificates from different issuers, so a chain can reach either an old or a new root; clients must pick a path that is valid. ## Who is this for - [IT ops](https://pkizerotohero.com/start/#itops) - [Developer](https://pkizerotohero.com/start/#developer) - [Expert](https://pkizerotohero.com/start/#expert) - [Leader](https://pkizerotohero.com/start/#leader) - [Beginner](https://pkizerotohero.com/start/#beginner) Level: **Practitioner** - [New to this? Start with The idea](https://pkizerotohero.com/handbook/cross-signing/#the-idea) - [Developer? Jump to How it really works](https://pkizerotohero.com/handbook/cross-signing/#how-it-really-works) - [IT ops? At work](https://pkizerotohero.com/handbook/cross-signing/#at-work) - [Leader? Why it matters](https://pkizerotohero.com/handbook/cross-signing/#why-it-matters-to-the-business) ## The idea - Beginner Root certificates have end dates too, usually 20 or 25 years away. Long before a root expires, its CA creates a new one. That raises a hard problem: a new root is useless until it is in people’s trust stores ([chapter 12](https://pkizerotohero.com/handbook/trust-stores/)), and getting it there takes years. Browsers update quickly, but old phones, smart TVs, payment terminals and forgotten servers may never receive an update. [Cross-signing](https://pkizerotohero.com/handbook/glossary/#cross-signing) bridges that gap. The established root signs a certificate for the new CA’s key. Now that one key has two certificates: - a **self-signed** one, which new trust stores contain; - a **cross-signed** one, issued by the old root, which old trust stores can follow. Every certificate signed by that key can be checked along either road. A new phone reaches the new root directly. An old phone walks one step further, to the old root it already trusts. Both arrive at a trust anchor, so both accept the certificate. There is a catch. The cross-signed certificate has its own end date, and so does the old root. When they expire, the old road closes. Devices that never received the new root start failing, even though nothing changed on the server. That is what happened to many older devices when the root that once cross-signed Let’s Encrypt expired. A second catch: some apps hard-code which keys or certificates they accept. This is called [pinning](https://pkizerotohero.com/handbook/glossary/#certificate-pinning). Pinning is meant to block fake certificates, but it also blocks real ones when a CA rotates its keys or roots, unless it was planned with care. In the game In the game: the Crossroads Inn In Level 14, Ken builds two roads from the Crossroads Inn to the castle: a new highway that new carriages know, and an old road that older carriages can still follow. When the old road’s bridge reaches its end date, Ken must make sure no traveller depends on it. Where the analogy stops being true: travellers do not see a map with both roads. Each client builds its own path from the certificates it receives and the roots it holds, and some clients pick a road that has already closed and give up instead of trying the other one. ## How it really works - Developer - Expert ### What exactly is a cross-signed certificate? It is a CA certificate with the **same subject name and the same public key** as another certificate, but a **different issuer** and therefore a different signature. Because the key is the same, the Subject Key Identifier is the same too ([chapter 13](https://pkizerotohero.com/handbook/certificate-chains/)). Any certificate signed by that key has an Authority Key Identifier that matches both versions, so a client can link it to either one. Let’s Encrypt’s live chain shows this. Here are two certificates for “ISRG Root X2”, one fetched from letsencrypt.org’s chain in October 2026 and one from a trust store: ```text subject=C=US, O=Internet Security Research Group, CN=ISRG Root X2 issuer=C=US, O=Internet Security Research Group, CN=ISRG Root X1 notAfter=Sep 2 23:59:59 2032 GMT SKI 7C:42:96:AE:DE:4B:48:3B:FA:92:F8:9E:8C:CF:6D:8B:A9:72:37:95 subject=C=US, O=Internet Security Research Group, CN=ISRG Root X2 issuer=C=US, O=Internet Security Research Group, CN=ISRG Root X2 notAfter=Sep 17 16:00:00 2040 GMT SKI 7C:42:96:AE:DE:4B:48:3B:FA:92:F8:9E:8C:CF:6D:8B:A9:72:37:95 ``` Same name, same key identifier, different issuers and different end dates. The first is ISRG Root X2 **cross-signed by ISRG Root X1**. The second is the **self-signed** ISRG Root X2 that newer trust stores contain. ### How does a client choose a path? A client collects candidate certificates from the server’s chain, its trust store, its cache and sometimes AIA downloads. Then it searches for a path from the leaf to any trust anchor ([RFC 4158](https://www.rfc-editor.org/rfc/rfc4158) describes the techniques). Two clients with different stores can therefore build different paths from the same server chain, and both can be right. letsencrypt.org sends four certificates: its leaf, the intermediate YE2, “Root YE” cross-signed by ISRG Root X2, and ISRG Root X2 cross-signed by ISRG Root X1. A modern trust store holds ISRG Root X2 itself, so OpenSSL stops there: ```text depth=0: CN=letsencrypt.org (untrusted) depth=1: C=US, O=Let's Encrypt, CN=YE2 (untrusted) depth=2: C=US, O=ISRG, CN=Root YE (untrusted) depth=3: C=US, O=Internet Security Research Group, CN=ISRG Root X2 ``` When we gave OpenSSL a trust store that holds **only** ISRG Root X1, like an older device, it used the cross-signed X2 certificate that the server sent and walked one step further: ```text depth=0: CN=letsencrypt.org (untrusted) depth=1: C=US, O=Let's Encrypt, CN=YE2 (untrusted) depth=2: C=US, O=ISRG, CN=Root YE (untrusted) depth=3: C=US, O=Internet Security Research Group, CN=ISRG Root X2 (untrusted) depth=4: C=US, O=Internet Security Research Group, CN=ISRG Root X1 ``` *Diagram: The leaf letsencrypt.org points to the intermediate YE2, which points to Root YE, which is cross-signed by ISRG Root X2. From there the path splits. Path A, for newer trust stores: ISRG Root X2 is found in the trust store, marked with a tick. Path B, for older trust stores that only hold ISRG Root X1: the cross-signed ISRG Root X2 certificate leads to ISRG Root X1, found in the trust store, marked with a tick.* One server chain, two valid paths. Each client ends at whichever root its own trust store holds. Not every client is this flexible. Older path builders took the first issuer they found and gave up if that path failed, even when another path would have succeeded. ### Why do roots expire, and what happens then? A root certificate has a `notAfter` date like any other certificate, and a cross-signed certificate can never usefully outlive its issuer for clients that check the issuer’s dates. When a root expires, clients that check it reject every path that ends there. Whether a certificate keeps working then depends on whether the client has, and finds, another valid path. Platforms differ in one detail that matters here. Let’s Encrypt notes that “Android intentionally does not enforce the expiration dates of certificates used as trust anchors” ([Let’s Encrypt: extending Android device compatibility](https://letsencrypt.org/2020/12/21/extending-android-compatibility/)). Many other clients, OpenSSL among them, do enforce them. ### What happened when DST Root CA X3 expired? When Let’s Encrypt started, its own root, ISRG Root X1, was in no trust store. IdenTrust’s established root, DST Root CA X3, cross-signed Let’s Encrypt so that its certificates worked everywhere from day one. Over the following years ISRG Root X1 reached most trust stores, but not old devices that no longer received updates. On 30 September 2021, DST Root CA X3, the older root that had cross-signed Let's Encrypt, expired, so devices that did not trust ISRG Root X1 began rejecting Let's Encrypt certificates. Three groups of clients behaved differently ([Let’s Encrypt](https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/)): - **Clients that trusted ISRG Root X1** and built paths well kept working: they ignored the expired road. - **Old Android devices** kept working because of “a special cross-sign from DST Root CA X3” that “extends past that root’s expiration”, which Android accepted because it does not enforce trust anchor expiry. - **Other old devices without ISRG Root X1**, and clients using OpenSSL 1.0.x, which Let’s Encrypt notes had “a quirk in certificate verification” with this chain, started to fail. That Android workaround had an end date of its own: Let's Encrypt's cross-signed chain for old Android devices expired on 30 September 2024 ([Let’s Encrypt: cross-sign expiration](https://letsencrypt.org/2023/07/10/cross-sign-expiration/)). The pattern is repeating. Let’s Encrypt’s newer roots are cross-signed in the same way: Root YE by ISRG Root X2, as the chains above show, and Root YR by ISRG Root X1 ([Let’s Encrypt: chains of trust](https://letsencrypt.org/certificates/)). Let's Encrypt lists both new roots as not yet included in the root programs' trust stores, so for now the cross-signs carry the trust. ### What is pinning, and why does it break rotation? An app that pins accepts a server only if one of the certificates in the path has a specific public key, usually identified by the SHA-256 hash of its SubjectPublicKeyInfo. Pinning narrows trust from “any CA in the store” to “these keys only”. The price is that every legitimate key change must be planned: - **Pin the leaf key**, and renewing with a fresh key breaks the app. - **Pin an intermediate**, and the CA’s routine switch to a new intermediate breaks it. - **Pin a root**, and a migration to a new root or a different CA breaks it. Pinning to a key, not a certificate, survives cross-signing, because both certificates hold the same key. The two ISRG Root X2 certificates above produce the same pin: ```text SPKI pin: diGVwiVYbubAI3RW4hB9xU8e/CH2GnkuvVFZE8zmgzI= SPKI pin: diGVwiVYbubAI3RW4hB9xU8e/CH2GnkuvVFZE8zmgzI= ``` Browsers have dropped HTTP Public Key Pinning ([RFC 7469](https://www.rfc-editor.org/rfc/rfc7469)), because one mistake could lock visitors out of a site for months. Mobile apps still pin. Android’s own guidance is to “always include a backup key”, and it allows an expiration date for pins, while warning that expiry “may enable attackers to bypass your pinned certificates” ([Android Network Security Configuration](https://developer.android.com/privacy-and-security/security-config)). ## At work - Developer - IT ops ### How do I see which roads my chain offers? Save the chain your server sends, split it into single certificates, and verify the leaf against different trust stores. These are the commands behind the two path listings above: ```bash openssl s_client -connect letsencrypt.org:443 -servername letsencrypt.org -showcerts /dev/null \ | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' > le-chain.pem csplit -s -z -f lecert- le-chain.pem '/-----BEGIN CERTIFICATE-----/' '{*}' cat lecert-01 lecert-02 lecert-03 > le-untrusted.pem # 1. With the default trust store openssl verify -show_chain -untrusted le-untrusted.pem lecert-00 # 2. With a store that holds only ISRG Root X1, like an older device openssl verify -show_chain -CAfile isrg-x1.pem -untrusted le-untrusted.pem lecert-00 ``` Both runs print `lecert-00: OK`, followed by the four-step and five-step paths. Build a small trust store file for each kind of old client you still support, such as an old Java version, an embedded device or an old phone, and run the second command against it before you change chains or CAs. ### How do I choose a chain? An ACME server may offer more than one chain for the same certificate. RFC 8555 lets it add “alternate” links, each of which “SHOULD express an alternative certificate chain starting with the same end-entity certificate” ([section 7.4.2](https://www.rfc-editor.org/rfc/rfc8555#section-7.4.2)). ACME clients let you choose, for example certbot’s `--preferred-chain` option, which picks the chain whose top certificate was issued by the name you give. What to serve: - the leaf and every intermediate, as always ([chapter 13](https://pkizerotohero.com/handbook/certificate-chains/)); - the cross-signed certificates that your oldest supported clients need, and no more; - never an expired cross-signed certificate, because some clients will try that road and fail. ### How do I keep trust stores current? The other half of root migration happens on the clients: - install operating-system updates, which carry root store updates; - update Java, because each JDK ships its own `cacerts`; - update `certifi` in Python projects and rebuild container images, whose CA bundles are frozen at build time; - upgrade Node.js, whose bundled roots are “fixed at release time”; - for devices in the field, make sure firmware updates can deliver new roots, years before the old ones expire. ### If I must pin, how do I do it safely? Pin public keys, not certificates, and always more than one. Compute a pin from any certificate like this: ```bash openssl x509 -in isrg-x2-self.pem -noout -pubkey | openssl pkey -pubin -outform der \ | openssl dgst -sha256 -binary | openssl enc -base64 ``` ```text diGVwiVYbubAI3RW4hB9xU8e/CH2GnkuvVFZE8zmgzI= ``` Then: - pin at least one backup key that is not in use yet, ideally from a different CA or root; - set an expiry date on the pins, and accept the trade-off Android describes; - keep a way to update the pins without shipping a new app, or be ready to ship one quickly; - write down which pins exist, so that the team that renews certificates knows about them. If a public website is all you protect, consider not pinning at all. Certificate Transparency monitoring ([chapter 17](https://pkizerotohero.com/handbook/certificate-transparency/)) catches mis-issued certificates without the risk of locking out your own users. ### What do failures look like? When a client follows a road that has closed, it reports an expired certificate somewhere in the chain, for example [`verify error:num=10:certificate has expired`](https://pkizerotohero.com/errors/openssl-certificate-has-expired/) at a depth above 0. When it cannot find any road to a root it trusts, it reports the familiar [`unable to get local issuer certificate`](https://pkizerotohero.com/errors/curl-unable-to-get-local-issuer-certificate/) or, in Java, [`PKIX path building failed`](https://pkizerotohero.com/errors/java-pkix-path-building-failed/). If only old devices fail, suspect the root, not the server. *Interactive tool: Chain checker (runs in the browser at [/tools/chain/](https://pkizerotohero.com/tools/chain/))* Paste your server’s chain to see each certificate’s issuer and end date. Look for cross-signed certificates near their end date, and for any that have already expired. ## Common mistakes - **“A certificate has exactly one chain.”** A certificate below a cross-signed CA has several valid paths. Each client builds its own. - **“When a root expires, everything under it stops working at once.”** Clients that have the newer root, or an alternative path, keep working. Old clients without one fail. - **“The client uses the chain exactly as the server sends it.”** Clients build their own path from what they receive and what they already hold, and may ignore some certificates. - **“Cross-signing means two CAs vouch, so trust is doubled.”** Each client trusts the certificate through one path only, ending at whichever root it holds. - **“Old devices will get the new root automatically.”** Many never update. Plan for them or decide to stop supporting them. - **“Pinning the leaf key is harmless.”** It breaks the app at the first renewal with a new key, unless a backup pin was shipped in advance. ## Why it matters to the business - Leader Why it matters to the business Root changes are slow outages for old devices When a CA’s old root expires, websites keep working in modern browsers but can stop working on old phones, payment terminals, set-top boxes and embedded devices that never update their trust stores. Customers on those devices see a broken service, often without a clear error. Ask which client platforms your services must support, and for how long. Make sure devices you ship can receive new roots, and that any certificate pinning in your apps is documented and has a tested backup plan. ## Check yourself Check yourself A server sends a chain that ends in a cross-signed certificate issued by an old root. A new phone has only the new root in its trust store. What happens? 1. It fails, because the chain points to the old root 2. It builds a path to the new root it trusts and accepts the certificate 3. It asks the user which root to use Show answer Answer B It builds a path to the new root it trusts and accepts the certificate What happened The phone accepts the certificate through the new root. Why The cross-signed and self-signed certificates share one key, so the phone can link the chain to the new root it already holds. The rule Clients build their own path to whichever trusted root they hold. Check yourself After an old root that once cross-signed a CA expires, which clients are most likely to fail? 1. Devices with recently updated trust stores 2. Old devices that never received the CA's new root 3. All devices at the same moment Show answer Answer B Old devices that never received the CA's new root What happened Old devices without the new root fail. Why Their only road to a trusted root ran through the expired certificate. Updated devices have a direct path. The rule Root expiry hits the clients that cannot update. Know which ones you support. Check yourself Your mobile app pins the SHA-256 hash of your server's leaf public key. At renewal, your ACME client generates a new key. What happens? 1. Nothing, because the certificate is still valid 2. The app rejects your server until the app is updated 3. The CA updates the pin automatically Show answer Answer B The app rejects your server until the app is updated What happened The app refuses the connection. Why The new certificate holds a new key whose hash does not match the pin built into the app. The rule Pin keys you control the rotation of, ship a backup pin, and plan every key change. ## Go deeper - [Let’s Encrypt: Chains of trust](https://letsencrypt.org/certificates/): the current roots, intermediates and cross-signs, a live example of everything in this chapter. - [Let’s Encrypt: DST Root CA X3 expiration](https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/): what changed and who was affected. - [Let’s Encrypt: extending Android device compatibility](https://letsencrypt.org/2020/12/21/extending-android-compatibility/): the cross-sign that outlived its own root. - [RFC 4158](https://www.rfc-editor.org/rfc/rfc4158): certification path building when several paths exist. - [RFC 8555, section 7.4.2](https://www.rfc-editor.org/rfc/rfc8555#section-7.4.2): alternate chains in ACME. - [Android Network Security Configuration](https://developer.android.com/privacy-and-security/security-config): pinning with backup keys and expiry. - [RFC 7469](https://www.rfc-editor.org/rfc/rfc7469): HTTP Public Key Pinning, now history, and a lesson in why pinning needs care. The crash course covers cross-signing in [lesson 6, Chains and trust stores](https://pkizerotohero.com/course/chains-and-trust-stores/). **Early edition.** Checked against the standards; an independent expert review is in progress. Found a mistake? [Tell us](https://pkizerotohero.com/contact/?topic=content-error&page=/handbook/cross-signing/). Last reviewed 20 October 2026Edition 0.9 ## Keep going Crash course: [Lesson 6: Chains and trust stores](https://pkizerotohero.com/course/chains-and-trust-stores/) ### Error pages for this chapter - [verify error:num=10:certificate has expired](https://pkizerotohero.com/errors/openssl-certificate-has-expired/) - [SSL certificate problem: unable to get local issuer certificate](https://pkizerotohero.com/errors/curl-unable-to-get-local-issuer-certificate/) - [PKIX path building failed … unable to find valid certification path to requested target](https://pkizerotohero.com/errors/java-pkix-path-building-failed/) ### How-to guides - [How to check the certificate chain a server sends](https://pkizerotohero.com/guides/check-remote-certificate-chain/) - [How to build the full chain file your server needs](https://pkizerotohero.com/guides/build-fullchain/) ### Tools - [Chain checker](https://pkizerotohero.com/tools/chain/): Paste a chain and see which link is missing, expired or out of order. - [Certificate decoder](https://pkizerotohero.com/tools/decoder/): Paste a PEM certificate and read every field in plain words. ### Topics - [Chains and trust stores](https://pkizerotohero.com/topics/chains-and-trust/)