paulmillr

Seven checks: verifying an AWS Nitro attestation in the browser My server says an enclave scored this borrower. The verify page checks that claim itself, against AWS's signature and Solana, without trusting my server. Seven checks, and where the obvious code would have been wrong. I'm building TEE Income Oracle. It reads a borrower's bank statement inside a sealed-off part of an AWS server, works out a credit tier (A, B or C), signs it, and writes it to Solana devnet. A lending pool reads the tier there before it lends. That sealed-off part is an AWS Nitro enclave: an isolated machine that nobody, including me as the operator, can log into or look inside while it runs. That's the point of using one. The bank statement goes in; only the tier comes out. But a lender has no reason to take my word that the tier came from an enclave, or that the enclave runs the code I published. So the enclave can produce proof, called an attestation document. AWS's hardware signs it, and it says two things: which code booted, and which signing key the enclave created for itself. If the document checks out, a tier signed with that key came from that code. The proof only helps if someone other than me checks it. So I wrote a verifier, a TypeScript library that also runs in the browser, on the app's verify page. You give it a borrower's wallet, it reads Solana directly, and it runs seven checks on that borrower's tier. It doesn't take my server's word for anything. This post is for developers who have heard of enclaves or "verifiable compute" and want to see what checking one actually involves. You don't need to know AWS Nitro, or any cryptography beyond "a signature proves who signed some bytes". I'll explain the rest as it comes up. What this post covers: - What's inside an attestation document, and how a tier on Solana leads back to one. - The seven checks, in plain words. - Four places where the obvious code would have been wrong: checking a re-packed copy instead of the signed bytes, rejecting a genuine AWS signature, failing every document a few hours after it was made, and a code fingerprint that changes when you edit a comment. - A gap in the verdict that had nothing to do with cryptography. - Why the browser checks a copy of the document, and why that's still safe. - What a pass doesn't prove, and a checklist if you verify attestations yourself. What's inside the document The document is a small binary file. Inside is a set of named fields, stored in CBOR, a compact binary format that does for binary data what JSON does for text. AWS signs that data and wraps it, with the signature, in a standard envelope called COSE_Sign1 (AWS describes the format here). Four fields matter for this post: | Field | What it means | | :------------------------ | :------------------------------------------------------------------------------------------------------------ | | pcrs | Fingerprints of what booted. Each PCR (platform configuration register) is a hash of one part of the software | | certificate, cabundle | The certificate whose key signed the document, and the chain of certificates that links it back to AWS | | timestamp | When the document was made | | public_key | A key the enclave generated when it started. My enclave signs tiers with it | My enclave runs on Marlin Oyster, a service that rents out Nitro enclaves. Oyster boots its own fixed base system, then starts my app from a docker-compose.yml file. PCR0, PCR1 and PCR2 fingerprint the base system. PCR16 fingerprints my compose file. Hashed together, those four give an image id: one value that names one exact build of the enclave. On Solana, a registry keeps one entry per enclave build: its image id, its signing key, a hash of its attestation document, and whether it has been retired. So from a borrower's wallet, the verifier can follow a chain back to AWS's hardware: 1. the wallet's tier on Solana, 2. the registry entry for the build that signed it, 3. that build's attestation document, 4. AWS's signature on that document. The seven checks | # | Check | What it asks | | :-- | :-------------------------------------------- | :------------------------------------------------------------------------------ | | 1 | Signed by AWS Nitro hardware | Is the signature valid, and does its certificate chain end at AWS's root? | | 2 | Marlin's published base image, not debug mode | Do PCR0–2 match Marlin's published values? (Debug mode zeroes them.) | | 3 | Runs our published code | Does PCR16 match the compose file at the release tag? | | 4 | Image id matches the registry | Do the four PCRs give the image id the registry recorded? | | 5 | Enclave key = on-chain attester | Is the document's key the one the registry says signs tiers? | | 6 | Same document the registry recorded | Does this document's hash match the one stored on Solana? | | 7 | Entry is active | Is the registry entry still active, and for the same type of proof as the tier? | The verifier runs all seven even after one fails, so a bad document shows everything wrong with it at once. AWS's root certificate, where every chain must end, is built into the verifier. It's never downloaded, because a verifier that downloads its root trusts whoever answers the download. A test checks the built-in copy against the fingerprint AWS publishes. 1. Check the bytes that were signed, not a re-packed copy A signature covers exact bytes. Two files can hold the same data and still differ byte for byte, and then the signature fits only one of them. That's the trap here. CBOR can store a set of fields in two ways: with the number of fields written at the start, or with an "end" marker after the last one. Nitro uses the second way. A CBOR library reads it fine, but when it packs the data back up, it uses the first. I tried it on one of my archived documents: its signed data, 4,478 bytes, became 4,477, and AWS's signature no longer matched. So the verifier decodes the data only to read its fields. It checks the signature over the bytes exactly as they arrived. The same rule applies to the certificates. A certificate's signature covers one section of the certificate, and the verifier checks that section exactly as it appears in the file, never a version it decoded and packed again. My tests nearly missed this. AWS's real certificates use DER, the strict standard encoding for certificates, so re-packing gives back the same bytes and both approaches pass. My AI code reviewer asked for a test that tells them apart. It takes a real certificate and writes one length field the long way round: the same certificate, but different bytes, and not the bytes AWS signed. A verifier that re-packs turns it back into the signed bytes and accepts it. One that checks the bytes as received rejects it, as it should. So re-packing fails in both directions: it rejects genuine documents, as the CBOR example shows, and it accepts bytes nobody signed. 2. A genuine AWS signature my library rejected The signature scheme here, ECDSA, has a quirk: every valid signature has a twin. From one valid signature, anyone can compute a second, different one that's also valid. Blockchains such as Bitcoin and Ethereum accept only one of the two, called the "low-S" form, so a transaction can't be given a second id. Standard ECDSA has no such rule. The library I use, noble-curves, applies it by default anyway. AWS has no reason to follow a blockchain rule, and a standard signer produces the other twin about half the time. One of my three archived documents has it, so with noble's default settings, a genuine AWS signature failed. With lowS: false, it passes. I checked both on the real document. The verifier uses the same setting for the certificate signatures in the chain. // sigStructure wraps the payload bytes exactly as they arrived. const valid = p384.verify( cose.signature, sha384(sigStructure(cose)), publicKeyPoint, { lowS: false, prehash: false }, ); Turning that rule off is safe here. The rule matters where a signature works as an id, like a transaction id. Here, the registry stores the hash of the whole document, signature included, so a copy carrying the twin signature fails check 6. 3. Check the certificates as of the document's date, not today's Certificates expire. AWS's guide says to check that today's date falls inside every certificate's validity period. That's right when you've just asked a running enclave to prove itself. My verifier checks archived documents, and the certificate that signs each one is valid for only about three hours. When I checked against today's date, all three archived documents failed. The oldest is from 2026-10-08, and some certificates in its chain had already expired. Checked against today, every verification would fail a few hours after the document was made. So the verifier checks the certificates as of the document's own timestamp: const leaf = verifyChain({ leaf: document.certificate, cabundle: document.cabundle, pinnedRoot: AWS_NITRO_ROOT_DER, atMs: document.timestampMs, }); That gives something up: the check no longer shows the document is recent. So that has to come from Solana instead. The document's hash was stored in the registry when the build was registered, right after a live check of the running enclave, and check 6 compares against it. The entry must also still be active (check 7). When the enclave restarts, it creates a new key and gets a new entry. By default, my rotation script then retires the old entry, and tiers it signed stop passing. I've made one exception. The devnet enclave stopped around 2026-10-22, and I believe the old machine and its key are gone, so I kept its entry active for the demo. That's an accepted risk, not a proof, and I plan to retire the entry once its last tiers are past their 30-day limit. 4. A code fingerprint that changes when you edit a comment Check 3 is what ties the running enclave to code anyone can read. Oyster computes PCR16 from the full text of the compose file it started from (the recipe is a few lines of hashing). Every byte counts, including comments and the newline at the end. Fix a typo in a comment, and the fingerprint no longer matches the running enclave. Two things follow. The compose file names my app's container image by its digest, a hash of the image itself, so the text points at exact code rather than a tag that can be moved. And the verifier reads the compose file from the build's release tag, not from the main branch, because main keeps changing. The deployment record stores the tag for exactly this reason. 5. Seven passes weren't the verdict My first version printed PASS when all seven checks passed. My AI code reviewer found the gap, and it had nothing to do with cryptography. The seven checks prove that the registry entry is genuine. They don't prove that the tier went through it. Other accounts can write records that look like tiers. What makes a tier trustworthy is who wrote it. The oracle program checks the enclave's signature before it writes a tier, and it always writes from one fixed address that only the program controls. A tier written from any other address never went through that check, however genuine the registry entry it points to. The first version even showed "UNEXPECTED signer" on screen, and still printed PASS. Now the writer is part of the verdict: export function verdictOf( checks: Check[], attestation: Pick<AttestationFacts, "signerOk">, ): Verdict { const allChecksPass = checks.every((check) => check.status === "pass"); return attestation.signerOk && allChecksPass ? "PASS" : "FAIL"; } The same review gave me a rule I now follow everywhere: never trust a value you can work out yourself. The verifier computes the program's fixed address from the program's id instead of reading it from a config file, and it checks that the program ids in the deployment file match the ones built into the app. 6. Why the browser checks a copy The enclave serves its attestation over plain HTTP. A page loaded over HTTPS can't fetch from plain HTTP: browsers block it as "mixed content". So the verify page ships the archived documents as part of the site, loaded only when needed. That sounds like "trust my copy", but it isn't. AWS signed the document, and its hash is stored on Solana. A changed copy fails check 6. It doesn't matter who delivers the bytes. I also dropped a planned option to fetch the document live from the enclave. Oyster's attestation server signs a fresh document for every request, so a live document never has the same hash as the archived one on Solana. A live document can confirm the key and the image id, but it can't replace the archive. What a PASS doesn't prove - That the code is correct. A pass proves which code ran, not that the code is right. Nothing has been audited yet. - That registration was honest. An admin key registers each build after checking its document off chain. Every registration is public, but someone holding a stolen admin key could register a key whose document would still pass. Beyond the demo, that should be a multisig, and later Solana itself should check the document. - That a lender should accept the tier today. A pass says where the tier came from. Whether it has expired, and whether it meets a pool's rules, are separate checks. - That the bank data was real. The demo bank is a sandbox: it speaks the real protocol, with demo keys. - That the RPC node told the truth. The page reads Solana through the public devnet endpoint and takes its answers as given. - That a retired machine's key is really gone. One build's entry stays active although its machine stopped (see section 3). If that key had leaked, tiers signed with it would still pass. If you verify attestations yourself - Check the signature over the bytes you received. Decode only to read fields; never re-pack before verifying. Write a test with an unusual but valid encoding, because real data won't catch the mistake. - Expect signatures in the "high-S" form if your ECDSA library rejects them by default, on the certificates as well as on the document. - Decide which date you check certificates against. Today, for a live attestation. The document's own date, for an archived one, and then show it's still current some other way. - Build the root certificate into your code. Don't download it. - Read measured files from the release tag. A fingerprint of a text file changes with every byte. - Check who wrote the claim, not just what it points to. Seven green checks on the wrong writer is still a fail. The verifier is about 1,600 lines of TypeScript. It doesn't use the browser's built-in crypto API, so the same code runs in Node and in the browser: verifier/. The verify page runs it in your browser. --- Originally published on prashantchaudhari.dev.

@kunchenguid Agree. In this case I have a much more polished app that you could find interesting. It’s for herdr. And it has features rex doesn’t. paulmillr/dispatch

The signed map and the app that rolls itself back > 📚 React Native Module Federation series — read it in full on warrendeleon.com, where new parts land first. Post 16 ended on the two problems it left open: "A remote that fell back is forgotten at the next launch, so a version that keeps failing fails again on every launch before its copy takes over. Remembering failures belongs with the other open problem: the map every binary trusts is still a plain text file anyone who can write to the bucket can change." Post 15 had said the second one in eight words: "The chunks are guarded; the map is naked." This post closes both, and the series with them. The map gets a signature, a release counter and, on the device, a memory. A map the app cannot verify is refused. An older map served again is refused. And when a signed map names a version that fails to load on two launches, the third launch does not try it: that remote runs from the copy in the binary while the other stays on the content delivery network (CDN), and nobody is paged. Post 1 named this as part of the integrity you build yourself: "you have to protect the choice of version too, so a replayed or rolled-back manifest can't quietly serve an old, vulnerable build." The first half is the signature and the counter: a second signing key, a string both sides rebuild byte for byte, and a number the device never lets go backwards. The second half is the failure count: a per-remote record of failed launches that survives the launch, a threshold, and the one event allowed to clear it. Start from post 16's finished state, the tag post-16-fallbacks; this post finishes at post-17-signing-rollback, the last tag of the series. You type the configuration changes and the smaller edits, and copy the rest from the end tag: the signing tools, the host's new and changed modules, its Jest config, and their tests. > 📊 Diagram: view it on warrendeleon.com Every refusal is handled where post 16 already catches a CDN that is not there. Two signatures, two jobs Post 14 signed every production chunk: Re.Pack's CodeSigningPlugin appends a JSON Web Token (a signed token in JavaScript Object Notation, or JSON) of the chunk's hash, made with an RSA (Rivest-Shamir-Adleman) key, and post 15 turned verification to strict, against the public half compiled into Info.plist and strings.xml. That key protects the content of every chunk. It says nothing about which chunks a binary should ask for. The map decides that, and post 15 wrote it as two remote-to-version entries in plain JSON. The map now carries two more fields: { "seq": 1, "versions": { "listApp": "1.2.0", "partyApp": "1.0.0" }, "sig": "..." } seq is the release counter, which the section "The counter" covers. sig is an Ed25519 signature (Ed25519 is an elliptic-curve signature algorithm), made by a second key the tools keep beside the chunk-signing one. The two keys have different jobs. The chunk key signs at build time, inside Re.Pack's plugin, and Re.Pack's native code verifies it as a script downloads. The map key signs at publish time, in the tools, and the host's own JavaScript verifies it at launch, before the launch registers a single remote at the versions the map names. Ed25519 fits that job: a 32-byte public key, a 64-byte signature, and verification that runs in plain JavaScript. Between them, the two signatures leave one gap. The map names versions, not their files, and a chunk's token signs its content hash and nothing about which version it belongs to. Someone who can write to the bucket can copy an older version's signed files into the directory of the version the map names, and neither check notices: each chunk's token is valid and its hash matches, and the map still names the version it named. Closing that gap means a map that binds each version to its files, which this series does not build. The signature is not over the JSON. Two tools can write the same map with different key orders or whitespace, and a signature over one of them fails against the other. It is over a string rebuilt from the values, the same way on both sides. Copy the shared signing module, the tool that uses it on one map, the CDN tool, which now signs every map it writes, and the key generator from the end tag, with their tests: npx degit@3.8.0 --force warrendeleon/react-native-module-federation#post-17-signing-rollback /tmp/pokedex-ref-17 cp /tmp/pokedex-ref-17/tools/{version-map-signing,sign-version-map,build-cdn,gen-signing-keys}.mjs /tmp/pokedex-ref-17/tools/{version-map-signing,build-cdn,gen-signing-keys}.test.mjs tools/ The string is in tools/version-map-signing.mjs: /** The bytes a signature covers, as a string. The versions must have passed versionsProblem. */ export function versionMapSigningInput(platform, appVersion, seq, versions) { const pairs = Object.keys(versions) .sort() .map(name => `${name}@${versions[name]}`) .join(','); return `${platform}|${appVersion}|${seq}|${pairs}`; } For the seq 1 map with listApp 1.2.0 and partyApp 1.0.0, on iOS, for app version 2.0.0, that is ios|2.0.0|1|listApp@1.2.0,partyApp@1.0.0. The platform and the app version are in it because the CDN holds one map per app version per platform, all signed with the same key. Without them, the map written for app 1.0.0 could be served at the web address (URL) of app 2.0.0's map with its signature intact, and a 2.0.0 binary would accept it and load the versions chosen for another binary. The host builds the same string with its own Platform.OS and APP_VERSION, so a map moved to another URL fails there as a forgery. The string has separators, so what goes between them is restricted. A remote name is letters, digits and underscores, starting with a letter; a version is the URL-safe shape post 15's parser already required. Without that rule a name holding @ or , could make two different maps produce the same string: {listApp, otherApp, partyApp} and {"listApp@1.2.0,otherApp": "3.0.0", partyApp} build the same bytes, so an edit that moved a boundary would keep the signature valid. The tools refuse to sign such a map, and the host refuses it before the verifier sees it. Hermes has no WebCrypto (the cryptography programming interface that browsers provide), so the host verifies with @noble/ed25519, a pure-JavaScript implementation. Its synchronous path needs a SHA-512 hash (Secure Hash Algorithm, 512-bit) that the library does not ship, so the host wires in @noble/hashes once, where the verifier module loads. Install both in the host at these versions: ( cd apps/host && npm install @noble/ed25519@3.2.0 @noble/hashes@2.4.0 ) import * as ed from '@noble/ed25519'; import { sha512 } from '@noble/hashes/sha2.js'; ed.hashes.sha512 = sha512 as typeof ed.hashes.sha512; Both packages ship only ECMAScript modules, the standard JavaScript module format, so they join transformIgnorePatterns in apps/host/jest.config.js; the host's whole Jest config comes from the end tag under "The counter", where the storage library is installed. The public half of the map key reaches the host as a TypeScript constant. tools/gen-signing-keys.mjs now makes the Ed25519 pair beside the RSA one, under the same rules post 14 set for the first key: generated only when neither half exists, an existing private key kept and never rotated, the public half derived from it when missing, a mismatched pair or an orphaned public key stops the script. Then it embeds the public halves: the chunk key into the plist and the strings file as before, and the map key, as the raw 32 bytes in base64url, into apps/host/src/shell/version-map-public-key.ts. Add that file now, with an empty value for the generator to fill: // --- The public half of the key that signs the version map. Written here by // tools/gen-signing-keys.mjs, the way the chunk-signing public key is written into Info.plist // and strings.xml: empty in the repository, because the key is generated per checkout, and filled // in by the one command a fresh clone runs before it builds. The value is the raw 32-byte Ed25519 // key in base64url, the form a JSON Web Key (JWK) carries it in. // // It can only verify, never sign, so it is safe to compile into every binary. Regenerating it for // an app that is already installed is not safe: a map signed by a new key does not verify against // the key those installs carry, and they treat every map as untrusted from then on. // --- export const VERSION_MAP_PUBLIC_KEY = ''; Run the generator in the checkout you built post 16 in. Post 14's chunk key is kept and was already embedded by the runs in posts 15 and 16, the map key is new, and one file gains a value: node tools/gen-signing-keys.mjs chunk-signing keypair already present, kept generated an Ed25519 map-signing keypair public key already embedded in apps/host/ios/Host/Info.plist public key already embedded in apps/host/android/app/src/main/res/values/strings.xml embedded the public key -> apps/host/src/shell/version-map-public-key.ts chunk-signing private key: <repo>/code-signing/private-key.pem (signs the chunks; never commit it, never ship it) chunk-signing public key: <repo>/code-signing/public-key.pem (what a host checks a chunk's signature against) map-signing private key: <repo>/code-signing/version-map-private.pem (signs the version map; never commit it, never ship it) map-signing public key: <repo>/code-signing/version-map-public.pem (what a host checks the map against) A fresh clone of the tag has no code-signing/, because the directory is ignored, so there the generator makes both pairs, embeds the chunk-signing public key in the plist and the strings file, and embeds the map-signing public key in version-map-public-key.ts. Update the comment in .gitignore above code-signing/, because the directory now holds two private keys: # Signing keys. The private keys sign every production chunk and the version map, and must never # be committed: this directory is ignored before `node tools/gen-signing-keys.mjs` is ever run. code-signing/ <blockquote class="callout callout-danger"> <p><strong>Never regenerate the map-signing key for an app that is installed, and never let either private key into the repository.</strong> Installed apps trust the key they were compiled with, and they refuse every map signed by another: a rotated key is a fleet that runs from its copies until a new binary ships. <code>code-signing/</code> is ignored before the generator ever runs, and the three files it writes the public halves into are committed empty, so a checkout carries slots and never a key. Before any commit, check that those three files are back to empty and that <code>git status</code> shows nothing under <code>code-signing/</code>.</p> </blockquote> The counter A signature proves who wrote the map. It does not prove when. Every map ever published carries a valid signature, so anyone who can serve a file to the app, by writing to the bucket or by sitting between the app and it, can serve a map from last month: validly signed, naming a version with a known fault. That is the replay post 1 warned about, and the counter is the answer. seq goes up with every publish. The device remembers the highest counter it has accepted and refuses any map below it, even one with a valid signature. Equal is accepted: the same map fetched twice is not an attack, and a device that refused it would run from its copies for no reason. A fresh install has accepted nothing, so it accepts any signed map; the counter protects a device that has already seen a newer one, and first-contact trust is the embedded key's job. The counter orders publishes, not versions. That is what lets a deliberate rollback through: a map at seq 3 that names listApp 1.1.0, published after seq 2 named 1.2.0, is a newer map naming an older version, and every device accepts it. An old map served again has a lower counter, whatever versions it names, and every device that has accepted a newer map refuses it. Post 15's one-line rollback survives; it gains a second step, signing the edited map with the next number. The device-side memory is MMKV, a key-value store the host reads through JSI (the JavaScript Interface React Native uses to call native code), synchronously, which keeps the boot gate and the two hooks that record a failed load and a CDN render free of extra awaits. It is a host dependency only, never shared with the remotes, and it holds two things: this counter, and the per-remote failure record built later in this post. Install it with the Nitro modules it is built on, then run pod install for the native side, the way post 13 did: ( cd apps/host && npm install react-native-mmkv@4.3.2 react-native-nitro-modules@0.37.1 ) ( cd apps/host/ios && bundle exec pod install ) Add apps/host/src/shell/storage.ts: import { createMMKV } from 'react-native-mmkv'; export const storage = createMMKV({ id: 'federation' }); /** The highest release counter accepted so far; zero before any map has been. */ export const VERSION_MAP_SEQ_KEY = 'mf.versionMap.seq'; /** The health record of one remote: a JSON RemoteHealth under this key. */ export function remoteHealthKey(remote: string): string { return `mf.health.${remote}`; } Under Jest, MMKV notices the test process and keeps its values in memory, so the launch code runs unchanged in the tests. Nitro fails first: importing MMKV imports Nitro, and Nitro looks its native module up the moment it loads and throws when there is none, before MMKV can notice anything. So jest.config.js maps react-native-nitro-modules to a stand-in in __mocks__ that exports the one name MMKV's test path still reads and never calls. The host's Jest config, the stand-in, and the modules and tests the rest of this post reads come from the end tag: cp /tmp/pokedex-ref-17/apps/host/src/shell/{scriptManager,remoteLocator,versionMapVerify,remoteHealth,storage}.ts /tmp/pokedex-ref-17/apps/host/src/shell/FederationBanner.tsx apps/host/src/shell/ cp /tmp/pokedex-ref-17/apps/host/App.tsx /tmp/pokedex-ref-17/apps/host/jest.config.js apps/host/ cp /tmp/pokedex-ref-17/apps/host/__mocks__/react-native-nitro-modules.js apps/host/__mocks__/ cp /tmp/pokedex-ref-17/apps/host/__tests__/{scriptManager,versionMapVerify,remoteHealth}.test.ts /tmp/pokedex-ref-17/apps/host/__tests__/{App,RemoteBoundary,FederationBanner}.test.tsx apps/host/__tests__/ The device must write the counter at the right point, not only with the right value. The map is verified in fetchVersionMap, but the device remembers its counter only in resolveFederation, after the map's versions have registered: try { registerCdnRemotes(versions); } catch (error) { console.warn('[federation] remotes could not be re-registered', error); mapVersions = {}; return runFromCopies('remotes could not be registered'); } // The map has been verified, read and its versions registered: now, and not before, it is the // map this launch runs on, and its counter is the one later launches measure against. A map // whose registration failed left the device's counter where it was. if (accepted.seq > readHighestAcceptedSeq()) { writeHighestAcceptedSeq(accepted.seq); } If the counter were written before registration, a map the launch never ran on would raise the minimum counter for every later map. The counter is one number for the whole CDN, not one per map. A device keeps the highest counter it accepted on any app version, and keeps it across a binary update: a map for app 1.0.0 published at seq 4 means a device that moves to app 2.0.0 refuses 2.0.0's map at seq 3. So a publish of any map has to move past all of them. The tools enforce the operator's half of that rule. tools/build-cdn.mjs refuses to run without MF_RELEASE_SEQ, and there is no default on purpose: a default would let a publish go out with a counter that has not moved. tools/sign-version-map.mjs, the tool that re-signs one edited map, reads the highest counter in every map under cdn-root/, on both platforms and every app version, and stops at a number that does not pass it: const highest = highestPublishedSeq(repoRoot); if (seq <= highest) { console.error( `The highest counter already published under cdn-root is ${highest}. MF_RELEASE_SEQ must be higher than every seq published before it, for any platform or app version, rollbacks included.`, ); process.exit(1); } <blockquote class="callout callout-warning"> <p><strong>The counter always goes up, rollbacks included.</strong> Rolling back is a new publish that names an older version, so it takes the next number like any other. A rollback signed with an old number is refused by every device that has seen a newer one, and those devices run from their copies, which is the opposite of what the rollback was for. The first break-it does the same with a rebuild: it re-signs the current map at an old number, on purpose.</p> </blockquote> Refuse and degrade The decision for one fetched map lives in apps/host/src/shell/versionMapVerify.ts, as a plain function, so the tests hand it a key they made and never touch the one in code-signing/. versionMapVerify.test.ts tests every refusal, the equal counter accepted, the higher counter naming older versions accepted, and a map signed for another app version or the other platform refused even though its signature is valid there. The function checks the shape first, in the order the fields are needed, then the names and versions, then the signature, then the counter, and every refusal names its reason: export type VersionMapRejection = | 'not an object' | 'no release counter' | 'no versions' | 'a remote name or version is not allowed' | 'no signature' | 'signature does not verify' | 'release counter is older than the last one accepted'; export function verifyVersionMap( raw: unknown, publicKey: string, identity: MapIdentity, highestAcceptedSeq: number, ): VerifyResult { if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) { return { ok: false, reason: 'not an object' }; } const { seq, versions, sig } = raw as Partial<Record<keyof SignedVersionMap, unknown>>; if (typeof seq !== 'number' || !Number.isInteger(seq) || seq < 0) { return { ok: false, reason: 'no release counter' }; } if (typeof versions !== 'object' || versions === null || Array.isArray(versions)) { return { ok: false, reason: 'no versions' }; } if (typeof sig !== 'string' || sig.length === 0) { return { ok: false, reason: 'no signature' }; } const entries = Object.entries(versions as Record<string, unknown>); if ( entries.some( ([name, version]) => !SAFE_NAME.test(name) || typeof version !== 'string' || !SAFE_VERSION.test(version), ) ) { return { ok: false, reason: 'a remote name or version is not allowed' }; } const signedVersions = Object.fromEntries(entries) as Record<string, string>; let valid = false; try { valid = ed.verify( base64ToBytes(sig), versionMapSigningInput(identity, seq, signedVersions), base64UrlToBytes(publicKey), ); } catch { // A signature or a key that is not even the right length throws rather than returning false. valid = false; } if (!valid) { return { ok: false, reason: 'signature does not verify' }; } if (seq < highestAcceptedSeq) { return { ok: false, reason: 'release counter is older than the last one accepted' }; } return { ok: true, map: { seq, versions: signedVersions, sig } }; } The signature is checked over the whole map, every remote it names included, because that is what was signed. Only then does post 15's parseVersionMap read out the remotes this binary knows. fetchVersionMap runs the two in that order and logs the reason when the first refuses: const verified = verifyVersionMap( (await response.json()) as unknown, VERSION_MAP_PUBLIC_KEY, { platform: Platform.OS, appVersion: APP_VERSION }, readHighestAcceptedSeq(), ); if (!verified.ok) { // Here the file arrived and failed a check. The warning names the check, so a launch that // runs on its copies says why. console.warn(`[federation] version map refused: ${verified.reason}`); return null; } A refusal returns null, and the caller treats that like every other way the map can fail: the launch runs from the copies, in post 16's bundled mode. A CDN the app cannot trust is treated like a CDN that is not there. Build the tree, with the first counter, and serve it: MF_RELEASE_SEQ=1 node tools/build-cdn.mjs npx http-server@14.1.1 cdn-root -p 8000 -c-1 --cors Every map build-cdn writes is signed, and its log says so: signed -> cdn-root/ios/maps/2.0.0/version-map.json (seq 1) Make the Release build on iOS, and the release build on Android with the emulator's address for the machine: ( cd apps/host && MF_CDN_BASE=http://localhost:8000 MF_APP_VERSION=2.0.0 npm run ios -- --mode Release ) ( cd apps/host && MF_CDN_BASE=http://10.0.2.2:8000 MF_APP_VERSION=2.0.0 npm run android -- --mode release ) Both boot with the banner reading cdn · listApp 1.2.0 · partyApp 1.0.0: the map verified, its counter was accepted, and the device now remembers 1. The next three steps show a tampered map refused, a replayed map refused and a signed rollback accepted, on iOS. On Android, repeat each step with android in place of ios. One counter serves both platforms and the numbers in this post count iOS alone, so on Android, and in every later step on either platform, sign with one more than the highest number already signed: sign-version-map refuses anything lower. An iOS Release build does not forward console.warn to the device log, so on iOS the evidence is the banner and the server's request log. On Android a release build writes the host's warnings to logcat, so each refusal's reason can be read: adb logcat -d -s 'ReactNativeJS:*' | grep '\[federation\]' First, tamper. Change the list's version in cdn-root/ios/maps/2.0.0/version-map.json from 1.2.0 to 1.1.0 by hand, keeping the rest of the file, and cold-start the app. The banner turns purple and reads bundled · listApp 1.2.0 · partyApp 1.0.0, the server's log shows one request, for the map, and nothing for either remote, and Android's logcat has the reason: [federation] version map refused: signature does not verify Second, replay. Put the file back as it was and keep a copy of it, because the seq 1 file is about to be the attacker's. Then publish it again with the next counter: MF_RELEASE_SEQ=2 node tools/sign-version-map.mjs ios 2.0.0 Cold-start: the map at seq 2 is accepted and the banner is blue again. Now put the seq 1 file back in its place and cold-start once more. Its signature is as valid as it was on the first launch, and the device refuses it anyway: [federation] version map refused: release counter is older than the last one accepted The banner is purple, both tabs run from their copies, and nothing of either remote was requested. Leave the seq 1 file where it is: the rollback step edits it and signs it again. Third, the rollback the counter allows. Edit the list's version to 1.1.0 again, and this time sign the edit: MF_RELEASE_SEQ=3 node tools/sign-version-map.mjs ios 2.0.0 Cold-start, and the app loads 1.1.0 from the CDN: the chip reads listApp 1.1.0, the banner cdn · listApp 1.1.0 · partyApp 1.0.0, and the server's log shows the 1.1.0 manifest and bundles requested. The map is signed, so the ordinary ship-a-version operation from post 15 has one more step: upload the version's directory, edit the line, sign with the next counter. The map is still written last, for the reason post 15 gave: a map that names a directory the CDN does not hold yet is a 404 for everyone who launches in between. In this post that window has a cost post 15 did not have: two failed launches roll the new version back, as the note on publishing order under "The third launch" explains. The map can be signed and wrong Everything so far guards the map's history. None of it guards the version's behaviour. A map can be current, validly signed and wrong: it can name a version that fails to load on every launch, because its manifest, container or chunks do not arrive or verify, or because its module throws as it is evaluated. Post 16's nets catch that once per session and run the copy, and the next launch asks the CDN again and fails the same way, on every launch, until an operator notices and publishes a different map. So the host keeps one small record per remote between launches, under its own key in the storage the counter uses: export interface RemoteHealth { /** The CDN version the failures were counted against. */ version: string; /** * How many launches have failed to load that version since it last rendered from the CDN. Only * a render of the CDN version in its tab clears the count. partyApp's boot loads in App.tsx can * add to it with no tab open. */ fails: number; } The decisions about the record are three functions in apps/host/src/shell/remoteHealth.ts, pure and tested without a device. A failure counts up for the same version and starts at one for a different one; a render clears the count, but only a render of the CDN version; and the threshold decides when the version stops being tried: export const FAILURE_THRESHOLD = 2; /** The record after a failed load. The same version counts up; a different one starts at one. */ export function nextHealthOnFailure(previous: RemoteHealth | null, version: string): RemoteHealth { return previous && previous.version === version ? { version, fails: previous.fails + 1 } : { version, fails: 1 }; } export function nextHealthOnSuccess( previous: RemoteHealth | null, version: string | undefined, renderedTheCopy: boolean, ): RemoteHealth | null { if (renderedTheCopy || !version) { return null; } if (previous && previous.version === version && previous.fails === 0) { return null; } return { version, fails: 0 }; } export function shouldRollBack( health: RemoteHealth | null, version: string, threshold: number = FAILURE_THRESHOLD, ): boolean { return health !== null && health.version === version && health.fails >= threshold; } The threshold is two. One failure can be a passing network problem: a dropped connection, a CDN edge that was slow once. Two is a pattern. Three would make users wait through one more failed attempt, up to the probe timeout for a manifest and up to Re.Pack's own script timeout for a container that stalls, to gain little confidence. A failed launch of a remote with a copy is not a launch spent on the broken version, because the net renders the copy; the cost of each extra failed launch is the wait, and the user's tab working on an older version meanwhile. The count is per version. A different version in the map starts the count over, so a fixed release is tried on its first launch, and a version with a long history of failures is ignored while the map names another one; its record is replaced at that remote's next failure or its next render from the CDN. And the count is of launches since the version last rendered from the CDN, not of launches in a row: a launch that never opens the tab cannot clear it, though partyApp's boot loads can still add to it, and a launch with the CDN out of reach runs the copies without asking and adds nothing. When the record is written The record is written at two moments, and the hard part of this post is choosing them. The failure is the easy one. Post 16's fallBack is where a remote drops to its copy in a CDN launch, from either net, and it now records the failure against the version the map named: function fallBack(remote: string): void { if (fallbackRemotes.has(remote)) { return; } fallbackRemotes.add(remote); console.warn(`[federation] ${remote} failed from the CDN; running its copy from the binary`); recordRemoteFailure(remote); setStatus({ ...status, versions: { ...status.versions, [remote]: bundledVersions[remote] }, embedded: [...status.embedded, remote].sort(), }); } // A remote dropped to its copy: the version the map named for it failed this launch. function recordRemoteFailure(remote: string): void { const version = mapVersions[remote]; if (!version) { return; } const health = nextHealthOnFailure(readHealth(remote), version); writeHealth(remote, health); console.warn( `[federation] ${remote} ${version} has failed ${health.fails} launch${health.fails === 1 ? '' : 'es'} since it last rendered from the CDN`, ); } The record leaves one case out on purpose. A component that loads and then throws as it renders is never counted. Post 16 decided that a remote whose load returned a component keeps its error state, because its code has run in this session, and Try again renders it again; a count here would send that remote to its copy at the third launch, which is a different decision from the one post 16 made and this post keeps. The success is the one to get right, and the rule is strict: only a render of the CDN version clears the count. A remote that fell back is rendering its copy, and the copy says nothing about the version the map names. Clearing on that render would reset a consistently broken version at every launch, and it would never reach the threshold. The event is reported from the host, by a component that mounts beside the remote inside the tab boundary's Suspense: return ( <Suspense key={this.state.attempt} fallback={<LoadingState />}> <Remote /> <LoadSuccessBeacon remote={remote} /> </Suspense> ); // Mounts once the remote beside it has rendered, and reports it once per mount. function LoadSuccessBeacon({ remote }: { remote: string }) { useEffect(() => { markRemoteLoadSuccess(remote); }, [remote]); return null; } A sibling inside the Suspense mounts in the same commit as the remote's first render and never before. A load that fails throws to the boundary and nothing in the Suspense commits; a component that throws as it first renders does the same. So the beacon reports that the remote painted. It does not report that the screen is correct: a component that renders and then fails later has already reported, so the beacon cannot detect a later failure. markRemoteLoadSuccess applies the rule, with the fallback set saying whether what rendered was the copy: export function markRemoteLoadSuccess(remote: string): void { if (!isRemoteName(remote)) { return; } const health = nextHealthOnSuccess( readHealth(remote), mapVersions[remote], fallbackRemotes.has(remote), ); if (health) { writeHealth(remote, health); } } The beacon lives in the host rather than in each remote. Health is the host's judgement of a remote, and a remote that knew about it could clear its own record. The remotes stay ignorant of how the host judges them, which is the same seam discipline post 7 argued for. The reads and writes around both moments are guarded: a record that cannot be read is no record, and a write that fails is logged and dropped. Nothing on the boot path threw before this post, and a storage failure is not a reason for a launch to refuse every remote or for a rendered tab to show its error state. The third launch The record is read in the boot gate, after the map is accepted and before anything registers. A remote whose named version is at the threshold, and which has a copy, is sent to its copy for the whole launch without trying the CDN: const rolledBack = REMOTE_NAMES.filter( name => hasEmbeddedCopy(name) && shouldRollBack(readHealth(name), versions[name]), ); rolledBack.forEach(name => { fallbackRemotes.add(name); console.warn( `[federation] ${name} ${versions[name]} failed ${FAILURE_THRESHOLD} launches since it last rendered from the CDN; running its copy from the binary without trying the CDN`, ); }); A rolled-back remote goes into post 16's fallback set, so the resolver and the manifest net serve the copy from the first script, and registerCdnRemotes registers it at the copy's manifest instead of the CDN's. A remote with no copy is tried from the CDN no matter how often it failed, because there is nothing else to run. The status records it twice: among the remotes running from a copy, and in rolledBack, so the banner can say which remotes failed this session and which the launch gave up on before trying. In a CDN launch a remote on its copy reads embedded when it dropped there after a failure, and rolled back when the launch sent it there. Watch the cycle on a map that is signed, valid and wrong. Point the 2.0.0 map at a version the CDN does not hold, 1.3.0, and sign it: MF_RELEASE_SEQ=4 node tools/sign-version-map.mjs ios 2.0.0 # or the next number, if Android was signed too Then cold-start three times, and read each launch from the banner, the server's log and, on Android, logcat: | Launch | The banner | The server's log | Android's logcat | |---|---|---|---| | 1 | cdn · listApp 1.2.0 embedded · partyApp 1.0.0 | the map, then the 1.3.0 manifest, 404 | listApp manifest returned 404, listApp failed from the CDN; running its copy from the binary, listApp 1.3.0 has failed 1 launch since it last rendered from the CDN | | 2 | the same | the same | the same, ending has failed 2 launches | | 3 | cdn · listApp 1.2.0 rolled back · partyApp 1.0.0 | the map, and nothing for listApp | listApp 1.3.0 failed 2 launches since it last rendered from the CDN; running its copy from the binary without trying the CDN | The first two launches show post 16's net: the manifest net drops the list to its copy mid-launch, and the tab opens on 1.2.0 behind its loading state. The third shows this post's rollback: the map still names 1.3.0, the device's record says that version failed twice, and the list makes no request. The party loaded from the CDN on all three. !The third launch on a map naming a version the CDN does not hold: the Pokédex tab lists Pokémon from Bulbasaur onwards under the chip listApp 1.2.0, and the blue banner above the tab bar reads cdn, listApp 1.2.0 rolled back, partyApp 1.0.0 Now recover from the bad version. Put the map back to 1.2.0 and sign it with the next number: MF_RELEASE_SEQ=5 node tools/sign-version-map.mjs ios 2.0.0 # or the next number The fourth launch tries 1.2.0 from the CDN, because the record is about 1.3.0 and a different version starts over; the list renders, the beacon reports it, and the record is cleared. The banner reads cdn · listApp 1.2.0 · partyApp 1.0.0 again. The rule set the version aside only until the map named a different version of listApp: nothing an operator did on the device brought it back, and nothing on the device needs undoing. <blockquote class="callout callout-info"> <p><strong>Two failed launches before a version's directory arrives keep that device on its copy until the map names another version.</strong> Post 15's rule stands: upload the version's directory first, then the map. In that order, no device reads a map that names a directory the CDN does not hold. In the other order, a device that launches twice in between counts two failed launches of the new version, and at its third launch it rolls back, whether the directory has arrived by then or not. It stays on the copy until the map names a different version, because a render of the copy never clears the count.</p> </blockquote> Now break it, twice The first break is the counter going backwards, and the way it happens is not an attack. tools/build-cdn.mjs seeds a CDN rather than operating one: it wipes a platform's tree and rebuilds it from the lists at the top of the file, signing every map at whatever MF_RELEASE_SEQ says. It checks that the counter is set, but never that it is higher than the one already published: a seeding tool has no previous release to move past. Rebuild the iOS tree with the number this post started on: MF_RELEASE_SEQ=1 node tools/build-cdn.mjs ios The tool signs every map at 1 and reports success. Cold-start the app. The banner is purple, bundled · listApp 1.2.0 · partyApp 1.0.0, the server's log shows only the map requested, and Android, after the same rebuild for its platform, logs release counter is older than the last one accepted. Every device that accepted 5, or whatever number came last, refuses 1, and it is the operator's own map they are refusing. Nothing on the device can be told the number went back on purpose, and that is the design: the counter is device state. A tool can only check what it can see. sign-version-map reads the highest number under cdn-root/, which the rebuild just rewrote, so it would sign 2 next without warning. The number has to live somewhere a rebuilt tree cannot forget: a release log, a tag, a variable in the pipeline that publishes. Rebuild past the highest number any device has seen and the next launch is blue again: MF_RELEASE_SEQ=6 node tools/build-cdn.mjs ios The second break is the beacon that trusts the copy, the last entry in the series' taxonomy of quiet failures: a health check that reports its own fallback as success. In markRemoteLoadSuccess, in apps/host/src/shell/scriptManager.ts, stop telling the rule whether what rendered was the copy: const health = nextHealthOnSuccess( readHealth(remote), mapVersions[remote], false, ); The host's tests catch the change first. scriptManager.test.ts has a case named "a render of the copy does not clear the count", and it fails. Rebuild the Release build with the change, point the 2.0.0 map at 1.3.0 again, as in the third-launch demo, sign it at the next counter, and cold-start the app three times. Every launch is launch one: the net drops the list to its copy, the record counts a failure, the copy renders, and the beacon clears the record moments later, because it no longer tells a render of the copy from a render of the CDN version. The third launch reads embedded and tries the CDN again, and so does every later launch, and Android's logcat ends every launch on listApp 1.3.0 has failed 1 launch since it last rendered from the CDN. Put the argument back, rebuild, and sign the map back to 1.2.0 with the next number. Run it These steps are for a device that has accepted nothing yet. The counter is kept in the app's data, so it survives a reinstall over the app. If your simulator or emulator ran this post's demos, delete the app from it first, or set MF_RELEASE_SEQ above the highest number it accepted, and sign from there. The keys, then the tree with its first counter, then the server: node tools/gen-signing-keys.mjs && MF_RELEASE_SEQ=1 node tools/build-cdn.mjs npx http-server@14.1.1 cdn-root -p 8000 -c-1 --cors The Release build on iOS, then Android with the emulator's address for the machine: ( cd apps/host && MF_CDN_BASE=http://localhost:8000 MF_APP_VERSION=2.0.0 npm run ios -- --mode Release ) ( cd apps/host && MF_CDN_BASE=http://10.0.2.2:8000 MF_APP_VERSION=2.0.0 npm run android -- --mode release ) With either running, edit one line of the platform's 2.0.0 map and sign it with the next counter. One counter serves both platforms, so a map signed for iOS at 2 means Android's next number is 3: MF_RELEASE_SEQ=2 node tools/sign-version-map.mjs ios 2.0.0 The suites, the smoke test and the tools' tests, which sign maps with keys they make for the run: ( for d in apps/host apps/list apps/party packages/ui; do ( cd "$d" && npx jest --silent ) || exit 1; done ) && sh scripts/federation-smoke.sh && node --test tools/*.test.mjs Before a commit, the three files the key generator fills go back to empty: git checkout the plist and the strings file, and set VERSION_MAP_PUBLIC_KEY back to ''. The private keys never leave code-signing/, and git status should never list that directory. What you built The demos in this post ran on an iOS Release build and an Android release build, against a local CDN, and the launch-by-launch table and the logged reasons are what those builds showed. Seventeen posts in, this is the whole thing: > 📊 Diagram: view it on warrendeleon.com Post 1 made its claims before a line of this existed. Each one has a post that built it, and the table says where a claim holds only in part: | Post 1 said | Where it became true | |---|---| | "A bug in one remote is a re-upload of that remote, not a store submission." | Post 14 built the artefact; post 15 shipped one with a directory and a line | | "The host also provides the heavy shared libraries once … so each remote consumes the host's copy instead of carrying its own." | Post 3, and the contract package in post 5 | | "shipping a copy of each remote inside the binary, so the reviewed app works on its own with no network, is an architecture you build" | Post 16, in part: the remotes' code runs with no CDN, and the Pokémon data and artwork still need the network | | "the app has to degrade to something safe instead of showing a blank screen" | Post 15 kept the process alive with one dead tab; post 16 gave the tab its copy; this post stops asking for a version that keeps failing | | "You have to sign what you ship and have the device verify it before executing" | Post 14 signed; post 15 turned verification to strict | | "protect the choice of version too, so a replayed or rolled-back manifest can't quietly serve an old, vulnerable build" | This post, for the map: an edited map and an older map are refused. Older signed files copied into a version's directory are not detected, as "Two signatures, two jobs" explains | What a real fleet still adds is named here and built nowhere in the series: a map that binds each version to its files; a staged rollout, so a new map reaches a fraction of devices first; telemetry, so an operator learns about a rolled-back remote from a dashboard rather than from a user; and key rotation, which this build forbids and a long-lived app eventually needs, through a binary that carries both keys for a while. Each is a post of its own in some other series. Post 1 asked whether this trade was worth making for your app. The table of its claims and the list of what a fleet still adds are the evidence for that decision. | Tag | Post | |---|---| | post-02-first-remote | Your first federated remote | | post-03-shared-singleton | The shared-singleton contract | | post-04-host-shell | The host shell: federated remotes as tabs | | post-05-contracts | The contract package | | post-06-shared-store | One shared store | | post-08-client-state | Client state across the seam | | post-09-tanstack-zustand | State stacks under federation | | post-10-two-backends | Two backends, one client? | | post-11-design-system | The design system as a federated singleton | | post-12-a11y-testing | Accessibility testing across federated remotes | | post-13-native-handoff | shell.navigateTo: native screens from a federated remote | | post-14-production-build | The production build and the three modes | | post-15-cdn-flip | CDN delivery: the version map, the resolver, and the live flip | | post-16-fallbacks | Fallback for federated remotes: the copy in the binary and the net in the session | | post-17-signing-rollback | This post | Every tag holds that post's finished state, and main ends at post-17-signing-rollback. Check any of them out and follow the series from that post. Sources - @noble/ed25519 — the pure-JavaScript Ed25519 used by the host; its README's "Enabling synchronous methods" is the hashes.sha512 wiring, with @noble/hashes/sha2.js as the hash - @noble/hashes — the SHA-512 the synchronous path needs - [RFC 8032: Edwards-Curve Digital Signature Algorithm (EdDSA)](rfc-editor.org

@SwiftOnSecurity @Ubiquiti @NextDNS So um you might wanna stop recommending Adguard paulmillr/encrypted-dns take a look around the repo particularly signed TLS profiles

we released chokidar 6.0.0 yesterday 👀 this was a pretty big rewrite by Paul which fixes a whole bunch of flakiness and Windows problems. the main win for me: that pesky EBADF error goes away now that it supports recursive watchers! 🎉

GitHub (url above is a basic redirect): paulmillr/dispatch Install: brew install --cask paulmillr/tap/dispatch

会社LINEグループ入れ系のメールで来るし、自己対策として1.1.1.2のプロファイルお守りとして入れといた。 ついでに家族のiphoneにも。 会社はCiscoのあれが入ってるけど、この惨状だと効果無いんだろうね。 paulmillr/encrypted-dns