rain6fish

AI Execution Governance, Part 9: Revocation Complete Is a Conclusion, Not a Status Field A side-effect row reads revoke_status = revoked. On the screen it becomes a green badge that says revoked. Those three words are not a field value. They are a conclusion, and a conclusion is supposed to be argued: did the compensation actually run, did the target actually return to its previous state, did the other system actually answer. One sentence: "revocation complete" is at least three claims, and folding them into one boolean replaces three unperformed checks with a badge. If that is enough, the rest is a table and three limits. 1. Completion is three things, not one - The compensation happened. The ledger says it ran. This is our own action and the easiest to prove, which is exactly why it gets mistaken for the conclusion. - The target returned. The business row is no longer in effect. The ledger and the business table are different places and they can disagree. - The outside answered. When the target lives in someone else's system, only that system can say it is gone. On our side the best we record is "the request arrived" or "it arrived and nothing came back". The three carry different evidence from different sources. Folding them into one boolean folds away which one is still missing. 2. What we return Our restore check returns three values: compensating → unknown revoked, target not soft-deleted → restored anything else → not_restored Two details matter. First, compensating is unknown, not "not revoked". It covers both "compensation is still running" and "it never started". Neither state knows the outcome, and writing either one down would be a claim. Second, the boolean view's false means unproven. We also expose a boolean, and it returns false for unknown as well. The comment on it says plainly that false means not proven restored, not known not revoked. Callers have to go back and read the three-valued state rather than treat the boolean as the answer. On the outside we have two words as well: possibly not delivered, and delivered, no answer. That second one is the absence of a conclusion, and we do not relabel it as completion. 3. Why two values are not enough Both ways of collapsing are wrong. - Collapse to "revoked" turns "I do not know" and "not revoked" into completion. This is what happens whenever the outside is unreachable. - Collapse to "not revoked" turns "in progress" into failure, so the revocation is retried or performed a second time. The middle value is not cautious wording. It is the only value that does not lie. A system with only two states will eventually assert something it does not know, on the first external timeout it meets. 4. A conclusion has to be refutable A badge that says revoked counts as evidence only if some observation could overturn it. We give the conclusion a refutation path. - Declared against held. What this call says it will revoke is compared against what the ledger actually holds, in both directions. A difference marks the group disputed and refuses to report the revocation complete. - The conclusion is blocked, not the write. When they disagree the write still completes. What is stopped is the sentence "it is done". That keeps non-deterministic tools from stalling on writes whose declarations legitimately differ from what is held. - The test runs backwards. Instead of asking whether two paths are independent, ask whether some fault can move one side and not the other. A break that moves both together means the agreement was never a check. 5. Four checks you can run 1. Which claims is this "complete" made of? If you cannot say, it is a status field. 2. Does the state machine have a middle value? Without unknown, it will lie on some timeout. 3. What does the boolean's false mean, refuted or unproven? Mixing the two is the dangerous part. 4. What observation would overturn this conclusion? If none, it was never verified. Closing Back to the badge. It should stand for three claims holding at once, not for three fields happening to line up. "Revocation complete" is not a status field. It is a conclusion that has to be argued, and when the argument cannot be made the right answer is unknown, not green. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The three-valued check is a way of reading here, not a shipped feature; KeelBase is its implementation and evidence carrier. Repository: rain6fish/KeelBase Three things stated plainly. The outside's "delivered, no answer" is a design value we have not tested end to end, so we cannot claim it was verified against a real external system. The declared-versus-held difference triggers only on the retry path that hits the idempotency key, because a first write has nothing to compare against. And the three-valued check is a reading of the code, without a bypass experiment. Drafted with an AI assistant. The system, the positions and the mistakes are mine.

AI Execution Governance, Part 9: Revocation Complete Is a Conclusion, Not a Status Field A side-effect row reads revoke_status = revoked. On the screen it becomes a green badge that says revoked. Those three words are not a field value. They are a conclusion, and a conclusion is supposed to be argued: did the compensation actually run, did the target actually return to its previous state, did the other system actually answer. One sentence: "revocation complete" is at least three claims, and folding them into one boolean replaces three unperformed checks with a badge. If that is enough, the rest is a table and three limits. 1. Completion is three things, not one The compensation happened. The ledger says it ran. This is our own action and the easiest to prove, which is exactly why it gets mistaken for the conclusion. The target returned. The business row is no longer in effect. The ledger and the business table are different places and they can disagree. The outside answered. When the target lives in someone else's system, only that system can say it is gone. On our side the best we record is "the request arrived" or "it arrived and nothing came back". The three carry different evidence from different sources. Folding them into one boolean folds away which one is still missing. 2. What we return Our restore check returns three values: compensating → unknown revoked, target not soft-deleted → restored anything else → not_restored Two details matter. First, compensating is unknown, not "not revoked". It covers both "compensation is still running" and "it never started". Neither state knows the outcome, and writing either one down would be a claim. Second, the boolean view's false means unproven. We also expose a boolean, and it returns false for unknown as well. The comment on it says plainly that false means not proven restored, not known not revoked. Callers have to go back and read the three-valued state rather than treat the boolean as the answer. On the outside we have two words as well: possibly not delivered, and delivered, no answer. That second one is the absence of a conclusion, and we do not relabel it as completion. 3. Why two values are not enough Both ways of collapsing are wrong. Collapse to "revoked" turns "I do not know" and "not revoked" into completion. This is what happens whenever the outside is unreachable. Collapse to "not revoked" turns "in progress" into failure, so the revocation is retried or performed a second time. The middle value is not cautious wording. It is the only value that does not lie. A system with only two states will eventually assert something it does not know, on the first external timeout it meets. 4. A conclusion has to be refutable A badge that says revoked counts as evidence only if some observation could overturn it. We give the conclusion a refutation path. Declared against held. What this call says it will revoke is compared against what the ledger actually holds, in both directions. A difference marks the group disputed and refuses to report the revocation complete. The conclusion is blocked, not the write. When they disagree the write still completes. What is stopped is the sentence "it is done". That keeps non-deterministic tools from stalling on writes whose declarations legitimately differ from what is held. The test runs backwards. Instead of asking whether two paths are independent, ask whether some fault can move one side and not the other. A break that moves both together means the agreement was never a check. 5. Four checks you can run Which claims is this "complete" made of? If you cannot say, it is a status field. Does the state machine have a middle value? Without unknown, it will lie on some timeout. What does the boolean's false mean, refuted or unproven? Mixing the two is the dangerous part. What observation would overturn this conclusion? If none, it was never verified. Closing Back to the badge. It should stand for three claims holding at once, not for three fields happening to line up. "Revocation complete" is not a status field. It is a conclusion that has to be argued, and when the argument cannot be made the right answer is unknown, not green. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The three-valued check is a way of reading here, not a shipped feature; KeelBase is its implementation and evidence carrier. Repository: rain6fish/KeelBase Three things stated plainly. The outside's "delivered, no answer" is a design value we have not tested end to end, so we cannot claim it was verified against a real external system. The declared-versus-held difference triggers only on the retry path that hits the idempotency key, because a first write has nothing to compare against. And the three-valued check is a reading of the code, without a bypass experiment. Drafted with an AI assistant. The system, the positions and the mistakes are mine.

AI Audit Evidence, Part 5: Five Sources in One File Do Not Check Each Other Has the same conclusion been answered by two paths that share nothing? Our evidence package puts five sources into one file: the authorization snapshot, the decision evidence, the audit row, the side-effect row and the operation audit row. A root digest pins them together. The package verifies offline and the signature matches. What that proves is that the five were signed together. It does not prove that they said anything to each other. A digest only answers whether the group was swapped out. One sentence: cross verification is not "find more sources". It is one conclusion answered independently by two non-sharing paths, with the system itself able to notice when they disagree. If that is enough, the rest is a table, a code fragment and three admissions. 1. What counts, and what is only packaging Three criteria. Miss one and it does not count. The two paths do not share a source. Not two views of the same data, and not the same code run twice. A disagreement is detectable. Code compares them and returns a distinguishable conclusion. Two values placed side by side, waiting for the reader to spot the conflict, does not count. It happens at runtime. A comparison that only exists in a test does not count, and neither does one a reader performs by hand. By those criteria, bundling five sources and hashing the bundle is packaging. Its value is real: the bundle cannot be altered without breaking the hash. But "cannot be altered" answers "was it changed", not "do the parts corroborate each other". Those two questions get conflated because both travel under the word integrity. 2. Our own account: two and a half This write was authorized. The second path recomputes from the policy version recorded in the row. What it compares is a single item, whether the tool was enabled. Role, writable field domain, destination and approval requirement are not recomputed. This write actually happened. The ledger, the business table and the audit row each carry a view. They are displayed side by side and never judged against each other. The revocation actually completed. The second path compares what this call declared against what the ledger holds, in both directions. This is the only place where a disagreement is detectable. The record was not altered. No second path in this deployment. The external anchor exists as a mechanism; this deployment has no receiver for it, so it is off rather than absent. The coverage statement holds. No second path. The conclusion is derived from one set of facts. Out of five conclusions, one meets all three criteria. The authorization row is half. A recomputation does happen, but for one of five checks. A reader sees "authorization was reviewed" and what was reviewed is a single item. Read that list as three grades: complete, meaning a second source that is actually compared; half, meaning a second source with one item compared; and none, meaning no second source at all. There is a fourth grade that is not on the list, and it costs the most: both sources are present and nobody compares them. We run two chains, an AI audit chain and an operation audit chain, and the same request lands in both. The material is there, the write points do not overlap, and it looks exactly like two independent paths. But no line of code pairs them by action. They are juxtaposed rather than corroborating. It is not on the list of five because we never treated "the two chains describe the same event" as a conclusion the package has to answer. The five are what the package claims. This one nobody claimed. The expensive kind of false cross is not missing material. It is material that nobody remembers to compare. 3. Why the one that counts, counts declared targets vs targets the ledger holds only declared → onlyDeclared only held → onlyStored either non-empty ⇒ mark the group disputed, refuse to report the revocation complete Independence: one side is what this call said it would write, the other is what the ledger already recorded. They are produced in different places, so an error in one does not automatically appear in the other. Detectability: a mismatch becomes a state, disputed, with the difference kept as evidence. Not two fields waiting to be noticed. The cost, stated plainly: it triggers in one situation, when a retry with the same arguments hits the idempotency key. 4. Something a reader asked out of us This section is about us. We published a five-layer evidence model: that a record exists, its source and integrity, its link to execution, coverage, and the verification of the result. A reader replied that a layer was missing. Some claims have no record at all. "This system complies with X", with nothing attached. He was right, and it was worse than he put it. Our first layer says that a record exists is not the same as the event happened, and it presupposes that a record exists. A claim with nothing attached sits outside the model: it neither satisfies nor violates it. It cannot enter. We added the layer, numbered L0. To be explicit: it was added after publication, and the published version does not contain it. If you go back to the article, you will not find it there. That exchange was itself a cross check. Our own conclusion was asked again by someone who does not share our view, in a different form, and the different form found the gap. The same question in the same wording returns the same blind spot. 5. Four checks you can run What is the second path for this conclusion? If you cannot name one, it is single source. Label it that way instead of calling it corroborated. Are the two paths independent, or is it the same data dressed differently? Bundling, merging and digesting all fall on the second side. When they disagree, does the system return a state, or does it place two values next to each other? Does that comparison run in production? If only in a test, it proves that test rather than the steady state. Closing Back to the file. It can prove it was not altered. It cannot prove the parts corroborate each other. To prove that, each conclusion has to name its second source, and the ones that cannot get labelled single source. The list will look bad. It will at least be accurate. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. Cross verification is a way of reading here, not a shipped feature; KeelBase is its implementation and evidence carrier. Repository: rain6fish/KeelBase Three things stated plainly. One of five conclusions is a complete cross check; the rest are half or nothing. The authorization row reviews a single item out of five. And L0 was added after publication, so anyone citing it should carry that sentence along. Drafted with an AI assistant. The system, the positions and the mistakes are mine.

Actions the Tool Registry Does Not Know About: Who Owns Them? One application, one events table, two rows. One row comes from your AI assistant: it calls create_event, a registered tool marked R3 that waits for your confirmation. The other comes from POST /api/v1/events: an ordinary HTTP route that touches no tool and writes nothing to the AI side-effect ledger. The two rows look the same. What differs is the entry, not the action. One sentence: what a governance system can vouch for is not what the registry contains, but which execution paths were verified and which were not — and the deliverable should be a list of unverified entries, not a coverage percentage. If you only want the conclusion, you can stop here. What follows is the four things that have to be separated, one entry-by-entry list, and a failure of our own; you can skim it. 1. A registry is a set; an entry is a path At boot this runtime registers 61 tools, all unconditional, into one Map. GET /ai/tools lists them with risk level and gate mode. That set answers "what do we intend to govern". It does not answer "where does the action enter". Three concrete places show this is not a matter of wording. First, there are two catalogues, and an auditor can only see one. GET /ai/tools maps the local registry only and requires read ToolInventory. External MCP tools are discovered at runtime, merged when tools are handed to the model, but absent from that list — they appear on /admin/mcp/tools, which requires manage all. An audit reader holds read only, so on the question "what can the agent call", the auditor can answer half of it; the other half needs an administrator, or is missed. Second, the gate does not treat "unregistered" as its own denial condition. When assertToolAllowed cannot find a tool, it catches the error and carries on; the comment says it plainly: let the later execute throw "not found". The executor does refuse: toolRegistry.execute calls getTool first and throws Tool "..." not found when the name is absent. So nothing here licenses a bypass — an unregistered name cannot reach the business layer. What it does license is a narrower claim: passing the gate is not the same as being on the registry. The gate checks a tool it found against its declaration; it does not reject what it did not find. Unregistered actions are held by the executor layer, and the two layers each mind their own business. Reading "the gate let it through" as "this action is registered" is exactly the merge this article is trying to take apart. Third, our own table drifted. The tool metadata table once had four labels pointing at tools that did not exist, while twelve real tools had none. The fix was not another hand-written list; it was to run the check against the live registry. That is our own incident, not a hypothetical: a hand-maintained table drifts silently the moment someone adds a tool and forgets it, or renames a tool and leaves the old entry behind. Raise the same class of error one level and you get this article's question: we have a gate that checks metadata against the registry, and no gate that checks the registry against the runtime's entry set. Nothing turns red when a new write path appears. 2. Four things that have to be separated - Action catalogue: what we declare we govern. Today: 61 registrations, one metadata table, plus externally discovered tools at runtime. - Execution entries: where an action actually starts — an HTTP route, an SSE stream, a WebSocket event, an MCP endpoint, a scheduled job, a workflow node, a queue consumer, a plugin route. - Governance coverage: which entries' writes really pass the same gate and the same executor. - Verification evidence: what proves the previous line — a source call site, a unit test, an end-to-end test, or "I read it once". Fold these four into a percentage and you get the usual "90% coverage". The problem is the denominator: the set of execution entries, which is exactly what we do not know. A percentage with an unknown denominator is a claim about a set you cannot enumerate. A list is different: it names each unverified entry, can be argued with, and can be closed. Put another way: coverage answers "how much did we do"; a list answers "what is left". Auditing needs the second. 3. How to audit the entries Start from the executor, not the registry. Find the few lines where a write actually happens, then walk backwards. In this repository the write executor is executeWrite, and the whole repository has exactly three call sites: two in the streaming chat loop (the trusted direct write, and the write after a confirmation is approved), and the R4 approval executor. Walking up from those three lines yields the SSE stream, WebSocket ai:chat, the admin streaming assistant, out-of-band decisions, and the governance-console callback. Then go the other way: for every remaining entry that can carry AI authorship, ask how its write happens. This step is reading source, not reading the registry. The thing most easily missed is not "is there a gate" but whether the gate is an optional argument: const execute = readOnlyExecutor ?? ((tool: string, args: Record<string, unknown>, uid: string) => toolRegistry.execute(tool, args, uid)); Plan and sub-agent steps are supposed to be read-only: with the read-only executor injected, write tools are refused. But the executor is optional; omit it and execution falls back to toolRegistry.execute — no gate, no confirmation, no side-effect ledger. The sub-agent has the same shape. We ran one call with no executor injected: the planner returned a write-tool step, and the tool name landed directly on registry.execute. Every call site in the current assembly injects the executor, so this is a path no one walks today — not a production bypass. But it is reproducible: a unit test even pins "no executor means direct registry call" as expected behaviour. One optional parameter turns "the gate" into "if the caller remembered". This method has a flaw it cannot fix itself: it is a snapshot. Add a controller tomorrow that calls toolRegistry.execute directly and nothing goes red. The closest thing to a runtime self-check today is /app/provenance, but it returns counts — total, read/write split, risk distribution. A count cannot name an entry. 4. Status, evidence level, reachability The status answers one question only: what is the conclusion. - verified — within the scope declared for this path, evidence shows it satisfies the stated governance condition: its write passes the gate and the executor, or an explicit refusal stops it. - failed — it is proven that at least one path violates that condition. - unknown — the evidence is insufficient to decide. Not counted as covered. - not_applicable — it is proven this path is outside the declared governance target, and the scope it belongs to is stated. A second question is not answered by those four values: on what ground. Folding "I read the source" and "I ran an end-to-end test" into one verified repeats exactly the merge this article criticises. So the evidence level is a separate field, independent of the status: - E1 — source read: the call chain reaches the gate and the executor at the execution point. - E2 — a unit or integration test exercises the path. - E3 — an end-to-end test, or a real run, verifies it. - E4 — a third party reproduces it from the steps in this article in an independent environment. (E0 means a claim with no evidence and is not used here.) verified / E1 and verified / E3 are not the same sentence: source proves "this call site calls executeWrite", not the runtime assembly, dependency injection, configuration switches or error branches; a passing test proves only the path it ran. The highest level in this piece is E3; nothing reaches E4. The entries, one by one — status, then evidence level, then whether the current assembly can reach it: - POST /ai/chat (non-streaming), /headless/chat, admin non-streaming. Write tools are refused, not executed; the gate is consulted at the start. No write occurs. Status verified (the refusal holds). Evidence E2. Reachable in the current assembly. - POST /ai/chat/stream, WS ai:chat, admin streaming. Writes go through executeWrite; the execution point re-checks the policy switch, field domain, destination and approval requirement, then claims the call. Audit row + claim row + side-effect ledger. Status verified. Evidence E1 (the entry-to-executor chain) plus E2 for the executor itself. Reachable. - POST /mcp (MCP exit). Write tools return "requires confirmation" and are not executed; executeToolForExternal applies the gate. A tool_call audit row is written with outcome=pending. Status verified (the refusal holds). Evidence E2. Reachable. - POST /admin/mcp/call (external MCP gateway). Calls the third-party server directly after the policy switch, R5 and confirmation checks. Writes an audit row; no local side-effect ledger. Status verified (gate yes, ledger no). Evidence E2. Reachable. - Confirmation and approval: in-conversation decision, out-of-band decision, governance-console callback. On approval, the write goes through executeWrite — the same execution point as streaming. Status verified. Evidence E2. Reachable. - Plan / sub-agent with the read-only executor injected. Write tools are refused; executeAgentRead enforces read-only. Audit is written by the caller. Status verified. Evidence E2. Reachable. - Plan / sub-agent with the executor omitted (the fallback). Calls toolRegistry.execute directly. No gate, no confirmation, no ledger. Status failed on the code structure. Evidence E1 plus the out-of-tree probe. Unreachable in the current assembly (every known call site injects the executor). - FLOW ai_task / human_task. One text generation / a flow_task row plus a notification. No tools involved. Status not_applicable (outside the AI tool surface). Evidence E1. - Scheduled AI (daily digest, 0 8 * * *). LLM output becomes a notification row. No tools, no tool_call row. Status not_applicable (writes a notification, not a business record). Evidence E1. - Queues (knowledge / push / reminder). Transcription, push, reminders. No tools. Status not_applicable (outside the AI tool surface). Evidence E1. - Direct business REST writes (e.g. POST /events). Calls the business service directly; no AI gate; operation audit, not the AI ledger. Status not_applicable (outside the AI entry surface). Evidence E1. - Plugin routes (POST /plugins/*). A plugin can fetch any host service and register a route. No gate, no ledger. Status unknown (not examined). Evidence E1 (the interface only). Reachability not examined. Two things about that list need saying. A note on the one failed, which has to be read as two sentences. Code structure: failed. With no executor injected, a write-tool name lands on registry.execute (whose real body is tool.execute), and gate, confirmation and ledger are all absent; this is reproducible, not inferred. Current assembly: no evidence that anything walks it. Every call site injects the executor — but that covers only the assembly points checked today, not a call site added tomorrow, and not runtime branches, so it does not amount to "safe". The two sentences must not be merged: writing "the code allows it" as "it happens in production" is one kind of dishonesty; writing "nobody walks it today" as "the code is fine" is the other. The reachability column exists for exactly this. not_applicable only says the path is outside the declared scope; it does not mean the business write is safe, and it does not mean KeelBase governs it. A direct business REST write and the AI tool write call the same service function (the create_event tool and POST /events both end at eventsService.create) and leave rows in the same table. It is not an AI entry, so the only promise this table makes about it is "we do not claim to govern it" — and if an AI path ever called it, the gate would not stand in the way. Reading not_applicable as "this write is already covered" turns out-of-scope into a pass. unknown cannot be read as verified. This runtime has a gap registry; each gap declares whether it makes the conclusion partial or unknown, and unknown means "we cannot even say what is missing", which is stronger than partial. A test pins that semantic: only a high-severity gap may produce unknown. The problem: the registry contains exactly two entries that describe this article's subject — risk-level-not-expressible-for-non-discrete-actions (an action surface that is not a discrete named tool cannot carry a risk level) and host-ai-path-not-governed (the host runs an AI path of its own that does not pass through the engine) — but this deployment's fact set pins both conditions to false, so a statement never declares them. We can name the gap; we cannot make it appear in the statement. That is slightly worse than not having the gap at all: it looks checked. The file gives a written reason for two of the pinned facts; these two have none. That is what an honest unknown looks like — not a blank, but "there is something here, and we have no input that can decide it". But this one is no longer an ordinary unknown; it is a defect in the statement mechanism itself. The two cases have to be recorded apart. An ordinary unknown is a missing evidence: not examined, or examined and undecidable, and another round of checking may clear it. This one is a missing mechanism: the gap is in the registry, its name can be said out loud, and yet the fact set pins the trigger to false, so the statement necessarily omits it and no amount of re-checking changes that. A statement that only distinguishes "implemented" from "not implemented" leaves this class permanently in the seam between the two. 5. Seven checks you can run 1. Which entry did this action come through? A client tap, an HTTP route, a WebSocket event, a scheduled job, a queue, a workflow node — write it down; do not write just "the AI". 2. Does that entry's write pass the same executor? Ask "is there a gate", not "is there an audit". An audit says it happened; a gate says it was allowed. 3. Does the same action produce the same row from two entries? If it does, the row alone cannot say who wrote it — unless you join the audit row and the effect row. 4. How many catalogues are there? Local registrations, the metadata table, externally discovered tools — and who reads each. 5. Where does unknown live? If the response has no value for unknown, absence will be read as a pass. 6. What goes red when a new entry appears? If the answer is "nothing", the coverage is hand-maintained. 7. What level of evidence backs this verified? Source reading and an end-to-end run both get written as "verified", and they vouch for very different amounts. 6. What we have, and what we do not What exists: 61 unconditionally registered tools with an auditor-readable inventory; a write executor that re-checks policy switch, field domain, destination, approval requirement and claim at the execution point; refusal of write tools on the MCP exit and the non-streaming entries; read-only enforcement for plan and sub-agent steps; a gap registry with unknown semantics and a verdict computed from facts; a completeness gate that checks tool metadata against the live registry. Boundaries: there are two catalogues (the local inventory omits externally discovered tools); the external MCP gateway has a gate and an audit but no local side-effect ledger (the target is someone else's); the plan/sub-agent fallback makes the gate optional; the gate does not refuse unregistered names; two gaps that should say "there is a path we cannot see" are pinned to false and never appear; a direct business write and an AI write share one service function and cannot be told apart from the row alone. What is not done: there is no runtime enumeration or assertion of entries — "the action surface is enumerable" remains a bet here, not a proven fact; no test proves "every AI entry is closed"; this table is a manual audit dated 2026-10-26, not a product feature; plugin routes and host-side AI paths have not been assessed. Keep three surfaces apart, or one of them will end up vouching for another. - The action surface: the 61 registered tools, the metadata table, the external MCP tools discovered at runtime — "what we intend to govern". - The entry surface: HTTP, SSE, WebSocket, MCP, scheduled jobs, queues, workflows, plugins that can trigger execution — "where the action enters". - The business side-effect surface: the write points that actually change business state — "what changed". Each can be incomplete on its own. A complete entry list does not imply a complete side-effect surface: a plugin can fetch any host service, and a general business service can be reached by a path nobody audited. So the conclusion should not stop at "enumerate the entries" — every business write inside the governed scope must complete only through a verified execution path, and every write outside that scope must have its boundary stated. The entry list is only an approximation of that sentence against today's code. Closing Back to the two rows: whether a write is governed depends on which entry it came through, not on which table it landed in. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The entry audit is a way of reading, not a shipped feature; KeelBase is its implementation and evidence carrier. Repository: rain6fish/KeelBase Four things stated plainly. Six entries carry E2 (one of them only at the executor layer), five carry only E1 (source), and the fallback row is E1 plus one out-of-tree run; nothing reaches E4. The fallback bypass was reproduced with a script that injects no executor; the production assembly does not walk it. The not_applicable calls for scheduled jobs, queues and workflows rest on "no tools are involved", not on a test that writes are impossible. And we do not write "all entries are covered", because nothing proves it. Responsibility is not in one place. The runtime must prove its own execution paths are actually governed — gate, confirmation, ledger, reproducible by test; the host application must identify and disclose the business entries that bypass the runtime, because only it knows where its writes come from; the integrator must confirm which business writes fall inside the governed scope, because scope is set at integration time. And the governance system must keep verified, failed, unknown and out-of-scope apart, rather than letting one tool catalogue vouch for the whole application. The registry says what we intend to govern; the entry list says what we actually reach. The difference between them is the governance boundary. Drafted with an AI assistant. The system, the positions and the mistakes are mine.

Actions the Tool Registry Does Not Know About: Who Owns Them? One application, one events table, two rows. One row comes from your AI assistant: it calls create_event, a registered tool marked R3 that waits for your confirmation. The other comes from POST /api/v1/events: an ordinary HTTP route that touches no tool and writes nothing to the AI side-effect ledger. The two rows look the same. What differs is the entry, not the action. One sentence: what a governance system can vouch for is not what the registry contains, but which execution paths were verified and which were not — and the deliverable should be a list of unverified entries, not a coverage percentage. If you only want the conclusion, you can stop here. What follows is the four things that have to be separated, one entry-by-entry list, and a failure of our own; you can skim it. 1. A registry is a set; an entry is a path At boot this runtime registers 61 tools, all unconditional, into one Map. GET /ai/tools lists them with risk level and gate mode. That set answers "what do we intend to govern". It does not answer "where does the action enter". Three concrete places show this is not a matter of wording. First, there are two catalogues, and an auditor can only see one. GET /ai/tools maps the local registry only and requires read ToolInventory. External MCP tools are discovered at runtime, merged when tools are handed to the model, but absent from that list — they appear on /admin/mcp/tools, which requires manage all. An audit reader holds read only, so on the question "what can the agent call", the auditor can answer half of it; the other half needs an administrator, or is missed. Second, the gate does not treat "unregistered" as its own denial condition. When assertToolAllowed cannot find a tool, it catches the error and carries on; the comment says it plainly: let the later execute throw "not found". The executor does refuse: toolRegistry.execute calls getTool first and throws Tool "..." not found when the name is absent. So nothing here licenses a bypass — an unregistered name cannot reach the business layer. What it does license is a narrower claim: passing the gate is not the same as being on the registry. The gate checks a tool it found against its declaration; it does not reject what it did not find. Unregistered actions are held by the executor layer, and the two layers each mind their own business. Reading "the gate let it through" as "this action is registered" is exactly the merge this article is trying to take apart. Third, our own table drifted. The tool metadata table once had four labels pointing at tools that did not exist, while twelve real tools had none. The fix was not another hand-written list; it was to run the check against the live registry. That is our own incident, not a hypothetical: a hand-maintained table drifts silently the moment someone adds a tool and forgets it, or renames a tool and leaves the old entry behind. Raise the same class of error one level and you get this article's question: we have a gate that checks metadata against the registry, and no gate that checks the registry against the runtime's entry set. Nothing turns red when a new write path appears. 2. Four things that have to be separated Action catalogue: what we declare we govern. Today: 61 registrations, one metadata table, plus externally discovered tools at runtime. Execution entries: where an action actually starts — an HTTP route, an SSE stream, a WebSocket event, an MCP endpoint, a scheduled job, a workflow node, a queue consumer, a plugin route. Governance coverage: which entries' writes really pass the same gate and the same executor. Verification evidence: what proves the previous line — a source call site, a unit test, an end-to-end test, or "I read it once". Fold these four into a percentage and you get the usual "90% coverage". The problem is the denominator: the set of execution entries, which is exactly what we do not know. A percentage with an unknown denominator is a claim about a set you cannot enumerate. A list is different: it names each unverified entry, can be argued with, and can be closed. Put another way: coverage answers "how much did we do"; a list answers "what is left". Auditing needs the second. 3. How to audit the entries Start from the executor, not the registry. Find the few lines where a write actually happens, then walk backwards. In this repository the write executor is executeWrite, and the whole repository has exactly three call sites: two in the streaming chat loop (the trusted direct write, and the write after a confirmation is approved), and the R4 approval executor. Walking up from those three lines yields the SSE stream, WebSocket ai:chat, the admin streaming assistant, out-of-band decisions, and the governance-console callback. Then go the other way: for every remaining entry that can carry AI authorship, ask how its write happens. This step is reading source, not reading the registry. The thing most easily missed is not "is there a gate" but whether the gate is an optional argument: const execute = readOnlyExecutor ?? ((tool: string, args: Record<string, unknown>, uid: string) => toolRegistry.execute(tool, args, uid)); Plan and sub-agent steps are supposed to be read-only: with the read-only executor injected, write tools are refused. But the executor is optional; omit it and execution falls back to toolRegistry.execute — no gate, no confirmation, no side-effect ledger. The sub-agent has the same shape. We ran one call with no executor injected: the planner returned a write-tool step, and the tool name landed directly on registry.execute. Every call site in the current assembly injects the executor, so this is a path no one walks today — not a production bypass. But it is reproducible: a unit test even pins "no executor means direct registry call" as expected behaviour. One optional parameter turns "the gate" into "if the caller remembered". This method has a flaw it cannot fix itself: it is a snapshot. Add a controller tomorrow that calls toolRegistry.execute directly and nothing goes red. The closest thing to a runtime self-check today is /app/provenance, but it returns counts — total, read/write split, risk distribution. A count cannot name an entry. 4. Status, evidence level, reachability The status answers one question only: what is the conclusion. verified — within the scope declared for this path, evidence shows it satisfies the stated governance condition: its write passes the gate and the executor, or an explicit refusal stops it. failed — it is proven that at least one path violates that condition. unknown — the evidence is insufficient to decide. Not counted as covered. not_applicable — it is proven this path is outside the declared governance target, and the scope it belongs to is stated. A second question is not answered by those four values: on what ground. Folding "I read the source" and "I ran an end-to-end test" into one verified repeats exactly the merge this article criticises. So the evidence level is a separate field, independent of the status: E1 — source read: the call chain reaches the gate and the executor at the execution point. E2 — a unit or integration test exercises the path. E3 — an end-to-end test, or a real run, verifies it. E4 — a third party reproduces it from the steps in this article in an independent environment. (E0 means a claim with no evidence and is not used here.) verified / E1 and verified / E3 are not the same sentence: source proves "this call site calls executeWrite", not the runtime assembly, dependency injection, configuration switches or error branches; a passing test proves only the path it ran. The highest level in this piece is E3; nothing reaches E4. The entries, one by one — status, then evidence level, then whether the current assembly can reach it: POST /ai/chat (non-streaming), /headless/chat, admin non-streaming. Write tools are refused, not executed; the gate is consulted at the start. No write occurs. Status verified (the refusal holds). Evidence E2. Reachable in the current assembly. POST /ai/chat/stream, WS ai:chat, admin streaming. Writes go through executeWrite; the execution point re-checks the policy switch, field domain, destination and approval requirement, then claims the call. Audit row + claim row + side-effect ledger. Status verified. Evidence E1 (the entry-to-executor chain) plus E2 for the executor itself. Reachable. POST /mcp (MCP exit). Write tools return "requires confirmation" and are not executed; executeToolForExternal applies the gate. A tool_call audit row is written with outcome=pending. Status verified (the refusal holds). Evidence E2. Reachable. POST /admin/mcp/call (external MCP gateway). Calls the third-party server directly after the policy switch, R5 and confirmation checks. Writes an audit row; no local side-effect ledger. Status verified (gate yes, ledger no). Evidence E2. Reachable. Confirmation and approval: in-conversation decision, out-of-band decision, governance-console callback. On approval, the write goes through executeWrite — the same execution point as streaming. Status verified. Evidence E2. Reachable. Plan / sub-agent with the read-only executor injected. Write tools are refused; executeAgentRead enforces read-only. Audit is written by the caller. Status verified. Evidence E2. Reachable. Plan / sub-agent with the executor omitted (the fallback). Calls toolRegistry.execute directly. No gate, no confirmation, no ledger. Status failed on the code structure. Evidence E1 plus the out-of-tree probe. Unreachable in the current assembly (every known call site injects the executor). FLOW ai_task / human_task. One text generation / a flow_task row plus a notification. No tools involved. Status not_applicable (outside the AI tool surface). Evidence E1. Scheduled AI (daily digest, 0 8 * * *). LLM output becomes a notification row. No tools, no tool_call row. Status not_applicable (writes a notification, not a business record). Evidence E1. Queues (knowledge / push / reminder). Transcription, push, reminders. No tools. Status not_applicable (outside the AI tool surface). Evidence E1. Direct business REST writes (e.g. POST /events). Calls the business service directly; no AI gate; operation audit, not the AI ledger. Status not_applicable (outside the AI entry surface). Evidence E1. Plugin routes (POST /plugins/*). A plugin can fetch any host service and register a route. No gate, no ledger. Status unknown (not examined). Evidence E1 (the interface only). Reachability not examined. Two things about that list need saying. A note on the one failed, which has to be read as two sentences. Code structure: failed. With no executor injected, a write-tool name lands on registry.execute (whose real body is tool.execute), and gate, confirmation and ledger are all absent; this is reproducible, not inferred. Current assembly: no evidence that anything walks it. Every call site injects the executor — but that covers only the assembly points checked today, not a call site added tomorrow, and not runtime branches, so it does not amount to "safe". The two sentences must not be merged: writing "the code allows it" as "it happens in production" is one kind of dishonesty; writing "nobody walks it today" as "the code is fine" is the other. The reachability column exists for exactly this. not_applicable only says the path is outside the declared scope; it does not mean the business write is safe, and it does not mean KeelBase governs it. A direct business REST write and the AI tool write call the same service function (the create_event tool and POST /events both end at eventsService.create) and leave rows in the same table. It is not an AI entry, so the only promise this table makes about it is "we do not claim to govern it" — and if an AI path ever called it, the gate would not stand in the way. Reading not_applicable as "this write is already covered" turns out-of-scope into a pass. unknown cannot be read as verified. This runtime has a gap registry; each gap declares whether it makes the conclusion partial or unknown, and unknown means "we cannot even say what is missing", which is stronger than partial. A test pins that semantic: only a high-severity gap may produce unknown. The problem: the registry contains exactly two entries that describe this article's subject — risk-level-not-expressible-for-non-discrete-actions (an action surface that is not a discrete named tool cannot carry a risk level) and host-ai-path-not-governed (the host runs an AI path of its own that does not pass through the engine) — but this deployment's fact set pins both conditions to false, so a statement never declares them. We can name the gap; we cannot make it appear in the statement. That is slightly worse than not having the gap at all: it looks checked. The file gives a written reason for two of the pinned facts; these two have none. That is what an honest unknown looks like — not a blank, but "there is something here, and we have no input that can decide it". But this one is no longer an ordinary unknown; it is a defect in the statement mechanism itself. The two cases have to be recorded apart. An ordinary unknown is a missing evidence: not examined, or examined and undecidable, and another round of checking may clear it. This one is a missing mechanism: the gap is in the registry, its name can be said out loud, and yet the fact set pins the trigger to false, so the statement necessarily omits it and no amount of re-checking changes that. A statement that only distinguishes "implemented" from "not implemented" leaves this class permanently in the seam between the two. 5. Seven checks you can run Which entry did this action come through? A client tap, an HTTP route, a WebSocket event, a scheduled job, a queue, a workflow node — write it down; do not write just "the AI". Does that entry's write pass the same executor? Ask "is there a gate", not "is there an audit". An audit says it happened; a gate says it was allowed. Does the same action produce the same row from two entries? If it does, the row alone cannot say who wrote it — unless you join the audit row and the effect row. How many catalogues are there? Local registrations, the metadata table, externally discovered tools — and who reads each. Where does unknown live? If the response has no value for unknown, absence will be read as a pass. What goes red when a new entry appears? If the answer is "nothing", the coverage is hand-maintained. What level of evidence backs this verified? Source reading and an end-to-end run both get written as "verified", and they vouch for very different amounts. 6. What we have, and what we do not What exists: 61 unconditionally registered tools with an auditor-readable inventory; a write executor that re-checks policy switch, field domain, destination, approval requirement and claim at the execution point; refusal of write tools on the MCP exit and the non-streaming entries; read-only enforcement for plan and sub-agent steps; a gap registry with unknown semantics and a verdict computed from facts; a completeness gate that checks tool metadata against the live registry. Boundaries: there are two catalogues (the local inventory omits externally discovered tools); the external MCP gateway has a gate and an audit but no local side-effect ledger (the target is someone else's); the plan/sub-agent fallback makes the gate optional; the gate does not refuse unregistered names; two gaps that should say "there is a path we cannot see" are pinned to false and never appear; a direct business write and an AI write share one service function and cannot be told apart from the row alone. What is not done: there is no runtime enumeration or assertion of entries — "the action surface is enumerable" remains a bet here, not a proven fact; no test proves "every AI entry is closed"; this table is a manual audit dated 2026-10-26, not a product feature; plugin routes and host-side AI paths have not been assessed. Keep three surfaces apart, or one of them will end up vouching for another. The action surface: the 61 registered tools, the metadata table, the external MCP tools discovered at runtime — "what we intend to govern". The entry surface: HTTP, SSE, WebSocket, MCP, scheduled jobs, queues, workflows, plugins that can trigger execution — "where the action enters". The business side-effect surface: the write points that actually change business state — "what changed". Each can be incomplete on its own. A complete entry list does not imply a complete side-effect surface: a plugin can fetch any host service, and a general business service can be reached by a path nobody audited. So the conclusion should not stop at "enumerate the entries" — every business write inside the governed scope must complete only through a verified execution path, and every write outside that scope must have its boundary stated. The entry list is only an approximation of that sentence against today's code. Closing Back to the two rows: whether a write is governed depends on which entry it came through, not on which table it landed in. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The entry audit is a way of reading, not a shipped feature; KeelBase is its implementation and evidence carrier. Repository: rain6fish/KeelBase Four things stated plainly. Six entries carry E2 (one of them only at the executor layer), five carry only E1 (source), and the fallback row is E1 plus one out-of-tree run; nothing reaches E4. The fallback bypass was reproduced with a script that injects no executor; the production assembly does not walk it. The not_applicable calls for scheduled jobs, queues and workflows rest on "no tools are involved", not on a test that writes are impossible. And we do not write "all entries are covered", because nothing proves it. Responsibility is not in one place. The runtime must prove its own execution paths are actually governed — gate, confirmation, ledger, reproducible by test; the host application must identify and disclose the business entries that bypass the runtime, because only it knows where its writes come from; the integrator must confirm which business writes fall inside the governed scope, because scope is set at integration time. And the governance system must keep verified, failed, unknown and out-of-scope apart, rather than letting one tool catalogue vouch for the whole application. The registry says what we intend to govern; the entry list says what we actually reach. The difference between them is the governance boundary. Drafted with an AI assistant. The system, the positions and the mistakes are mine.

What Can One AI Audit Record Actually Prove? An audit record is on the table: one confirmation, one execution, one side effect. The hash chain verifies. What does that prove? One sentence: one record does not prove all five layers: a valid signature reaches only the second, and coverage and the external result are separate checks. Most people read it as two things: the event happened, and the record is true. There are five different questions inside that sentence, and one record usually answers only one or two of them. This piece gives a five-layer evidence model. If you only want the conclusion, you can stop here. The rest is the failure modes and the five claims behind that sentence, and you can skip it. The bold sentence in each section carries the point. 1. Five questions, not five fields A record can answer five questions, and they are not the same thing: Event record: what did the system record? Source and integrity: who produced the record, and was it altered? Execution linkage: does this record correspond to the path that actually ran? Coverage: which execution paths were verified? Result verification: what actually happened to the business side effect? The layers build on each other, but they do not upgrade automatically. A higher layer cannot be inferred from a lower layer's fields. A signature does not add coverage. A coverage statement does not verify an external result. 1. What one real record looks like This is a record from a local run on 2026-10-15. The tool is create_followup_task, risk level R3, approved from the pending list: confirmation #3 create_followup_task R3 approved decided 05:54:37.327 claimed .332 executed .365 error null audit #27 action=tool_call tool=null outcome=null is_error=0 detail="create_followup_task({\"customerId\":1,\"title\":\"Followup Task\"})" hash=b0d6bf… prev_hash=565b3f… effect #1 crm_task:10 before_snapshot=null after_snapshot={"id":10,...} revoke_class=local_compensate The three rows together are "one operation". But there is a non-obvious detail: audit row #27 has tool and outcome empty. The tool name exists only inside the detail string, and the result can only be inferred from is_error=0. A later row in the same database (#37, later that day) has tool=create_followup_task and outcome=executed. The same kind of record can look different at different times. The record is not false. It is authentic, on the chain, and the hashes are continuous. It simply does not explain itself. A reader who does not know the detail convention cannot say what ran or what the result was. 1. Reading the record layer by layer Layer 1, event record: three rows exist, a confirmation, an audit row and a side-effect row. It does not prove the event happened as recorded; the record itself can be wrong. Layer 2, source and integrity: the audit row is on the chain, with continuous hash and prev_hash. It does not prove the tail was not deleted or that nothing was omitted. The confirmation and side-effect rows are not on this chain. Layer 3, execution linkage: the confirmation's execution axis is executed, and the effect is crm_task:10. It does not prove this effect was produced by this confirmation; the link is by tool, argument hash and time, not one foreign key. Layer 4, coverage: the package's coverage statement can be consulted separately. One record does not say other execution paths were covered. Layer 5, result verification: after_snapshot is present. It does not make the change comparable, because before_snapshot is null, and it does not prove the external target's state. The strongest layer in this record is layer 2. The weakest is layer 5. It is a real record, not a complete piece of evidence. 2. Two counterexamples Counterexample A: a valid signature over an incomplete record. Audit row #27 is on the chain, with continuous hash and prev_hash. But its tool and outcome are empty, and the side-effect row has no before_snapshot. So: you can prove the audit row was not altered you cannot use this record alone to say which tool ran, what the result was, or what changed before it A signature authenticates bytes, not meaning. A real, signed record can still fail to answer the question it appears to answer. Counterexample B: a complete record with an unknown business result. This is a design-level scenario, not a test of the record above. A write is sent to an external system. The local side-effect row says executed, but the external system is unreachable and has no query endpoint. The correct conclusion is unknown, not success. A complete local record only says the local side is complete. An unverified external state cannot be written as success. 1. Not a field checklist The five layers are not a field checklist. They are five refutable claims: L1: a record exists is not the same as the event happened. L2: a valid chain or signature is not the same as no omissions and no tail deletion. L3: a record is linked to an execution is not the same as the execution matched its authorization. L4: one action was covered is not the same as every path was covered. L5: a local result was recorded is not the same as the external business state was verified. These claims do not try to prove that AI never makes mistakes. They ask which guarantees a system can offer when AI misreads, overreaches, retries, times out or hits an exception, and how far each guarantee can be verified. They also explain why adding fields does not add guarantees: a signature at L2 does not make coverage at L4 true, and coverage at L4 does not make an external result at L5 verified. Each layer needs its own evidence. The test of these claims is not more terminology. The test is whether they can point at a counterexample, narrow a guarantee, or turn a vague promise into a verifiable criterion. All five are candidates, not proven theorems. 1. What we have, and what we do not What exists: the confirmation's execution axis (decided_at, execution_claimed_at, executed_at, execution_error); the audit hash chain; side-effect rows with before and after snapshots; the coverage statement; the audit reader role. Boundaries: the record above already shows three. Structured fields can be empty. A side-effect row can lack a before snapshot. The confirmation and side-effect rows are not on the audit chain. Beyond that: without an external anchor, tail completeness can only be unknown; an unreachable external result is unknown; the operation audit row scope is not yet narrowed by organisation. What is not done: the five-layer model is not an API or a product feature. It is a way to read evidence, which you can apply to any agent runtime without waiting for us to ship five endpoints. 1. Five checks you can run Which layer does this record answer? "Recorded" or "verified"? What proves integrity? A chain, a signature, a coverage statement, or only a sentence? What links the record to the execution? One id, or tool plus argument hash plus time? Can someone else rebuild the link? Where is coverage written? How are uncovered paths expressed: unknown, absent, or pretended complete? How far does result verification go? When the external state is unknown, is the conclusion unknown? Closing Back to the opening: what a record proves depends on which layer you are asking about. What we do: split evidence into five layers and say what each cannot prove; show a real record where "authentic but incomplete" is visible; keep external results and tail completeness as unknown instead of writing them as success. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The five-layer model is a way to read evidence, not a shipped feature; KeelBase is its implementation and evidence carrier. Repository: rain6fish/KeelBase Four things stated plainly. The record above comes from a local run, not production data. The five-layer model is a method, not a shipped feature. External results and tail completeness remain unknown, and the operation audit row scope is still an open item. The five layers are candidates, and counterexamples are welcome. An honest record does not claim all five layers are full. It says where each layer stops. Drafted with an AI assistant. The system, the positions and the mistakes are mine.

What Can One AI Audit Record Actually Prove? An audit record is on the table: one confirmation, one execution, one side effect. The hash chain verifies. What does that prove? One sentence: one record does not prove all five layers: a valid signature reaches only the second, and coverage and the external result are separate checks. Most people read it as two things: the event happened, and the record is true. There are five different questions inside that sentence, and one record usually answers only one or two of them. This piece gives a five-layer evidence model. If you only want the conclusion, you can stop here. The rest is the failure modes and the five claims behind that sentence, and you can skip it. The bold sentence in each section carries the point. 1. Five questions, not five fields A record can answer five questions, and they are not the same thing: Event record: what did the system record? Source and integrity: who produced the record, and was it altered? Execution linkage: does this record correspond to the path that actually ran? Coverage: which execution paths were verified? Result verification: what actually happened to the business side effect? The layers build on each other, but they do not upgrade automatically. A higher layer cannot be inferred from a lower layer's fields. A signature does not add coverage. A coverage statement does not verify an external result. 2. What one real record looks like This is a record from a local run on 2026-10-15. The tool is create_followup_task, risk level R3, approved from the pending list: confirmation #3 create_followup_task R3 approved decided 05:54:37.327 claimed .332 executed .365 error null audit #27 action=tool_call tool=null outcome=null is_error=0 detail="create_followup_task({\"customerId\":1,\"title\":\"Followup Task\"})" hash=b0d6bf… prev_hash=565b3f… effect #1 crm_task:10 before_snapshot=null after_snapshot={"id":10,...} revoke_class=local_compensate The three rows together are "one operation". But there is a non-obvious detail: audit row #27 has tool and outcome empty. The tool name exists only inside the detail string, and the result can only be inferred from is_error=0. A later row in the same database (#37, later that day) has tool=create_followup_task and outcome=executed. The same kind of record can look different at different times. The record is not false. It is authentic, on the chain, and the hashes are continuous. It simply does not explain itself. A reader who does not know the detail convention cannot say what ran or what the result was. 3. Reading the record layer by layer Layer 1, event record: three rows exist, a confirmation, an audit row and a side-effect row. It does not prove the event happened as recorded; the record itself can be wrong. Layer 2, source and integrity: the audit row is on the chain, with continuous hash and prev_hash. It does not prove the tail was not deleted or that nothing was omitted. The confirmation and side-effect rows are not on this chain. Layer 3, execution linkage: the confirmation's execution axis is executed, and the effect is crm_task:10. It does not prove this effect was produced by this confirmation; the link is by tool, argument hash and time, not one foreign key. Layer 4, coverage: the package's coverage statement can be consulted separately. One record does not say other execution paths were covered. Layer 5, result verification: after_snapshot is present. It does not make the change comparable, because before_snapshot is null, and it does not prove the external target's state. The strongest layer in this record is layer 2. The weakest is layer 5. It is a real record, not a complete piece of evidence. 4. Two counterexamples Counterexample A: a valid signature over an incomplete record. Audit row #27 is on the chain, with continuous hash and prev_hash. But its tool and outcome are empty, and the side-effect row has no before_snapshot. So: you can prove the audit row was not altered you cannot use this record alone to say which tool ran, what the result was, or what changed before it A signature authenticates bytes, not meaning. A real, signed record can still fail to answer the question it appears to answer. Counterexample B: a complete record with an unknown business result. This is a design-level scenario, not a test of the record above. A write is sent to an external system. The local side-effect row says executed, but the external system is unreachable and has no query endpoint. The correct conclusion is unknown, not success. A complete local record only says the local side is complete. An unverified external state cannot be written as success. 5. Not a field checklist The five layers are not a field checklist. They are five refutable claims: L1: a record exists is not the same as the event happened. L2: a valid chain or signature is not the same as no omissions and no tail deletion. L3: a record is linked to an execution is not the same as the execution matched its authorization. L4: one action was covered is not the same as every path was covered. L5: a local result was recorded is not the same as the external business state was verified. These claims do not try to prove that AI never makes mistakes. They ask which guarantees a system can offer when AI misreads, overreaches, retries, times out or hits an exception, and how far each guarantee can be verified. They also explain why adding fields does not add guarantees: a signature at L2 does not make coverage at L4 true, and coverage at L4 does not make an external result at L5 verified. Each layer needs its own evidence. The test of these claims is not more terminology. The test is whether they can point at a counterexample, narrow a guarantee, or turn a vague promise into a verifiable criterion. All five are candidates, not proven theorems. 6. What we have, and what we do not What exists: the confirmation's execution axis (decided_at, execution_claimed_at, executed_at, execution_error); the audit hash chain; side-effect rows with before and after snapshots; the coverage statement; the audit reader role. Boundaries: the record above already shows three. Structured fields can be empty. A side-effect row can lack a before snapshot. The confirmation and side-effect rows are not on the audit chain. Beyond that: without an external anchor, tail completeness can only be unknown; an unreachable external result is unknown; the operation audit row scope is not yet narrowed by organisation. What is not done: the five-layer model is not an API or a product feature. It is a way to read evidence, which you can apply to any agent runtime without waiting for us to ship five endpoints. 7. Five checks you can run Which layer does this record answer? "Recorded" or "verified"? What proves integrity? A chain, a signature, a coverage statement, or only a sentence? What links the record to the execution? One id, or tool plus argument hash plus time? Can someone else rebuild the link? Where is coverage written? How are uncovered paths expressed: unknown, absent, or pretended complete? How far does result verification go? When the external state is unknown, is the conclusion unknown? Closing Back to the opening: what a record proves depends on which layer you are asking about. What we do: split evidence into five layers and say what each cannot prove; show a real record where "authentic but incomplete" is visible; keep external results and tail completeness as unknown instead of writing them as success. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The five-layer model is a way to read evidence, not a shipped feature; KeelBase is its implementation and evidence carrier. Repository: rain6fish/KeelBase Four things stated plainly. The record above comes from a local run, not production data. The five-layer model is a method, not a shipped feature. External results and tail completeness remain unknown, and the operation audit row scope is still an open item. The five layers are candidates, and counterexamples are welcome. An honest record does not claim all five layers are full. It says where each layer stops. Drafted with an AI assistant. The system, the positions and the mistakes are mine.

An Auditor Is Not an Admin: Giving Someone the Evidence Is Not Giving Them the System An auditor has to review what an AI did. The cheapest way to let them do that is to hand them an admin account. But then the person they are supposed to review becomes themselves. This piece is about building a read-only auditor role: its reach is an explicit list, every field is rebuilt from a whitelist, row scope comes from the account rather than the request, and the signed evidence stays whole and unprojected. 1. Admin is the cheapest answer, and the wrong one An audit reader's job is to review what someone else did. They need evidence, not operational authority. But in most systems read access follows write access: whoever can manage everything can obviously read everything. So the shortest path to "let them see the audit" is to give them admin. The cost is that they also gain the ability to change policy, approve requests and execute tools, which is exactly what they came to review. What is needed is not a weaker admin. It is a role whose reach can be enumerated item by item. If you cannot enumerate it, it is still an admin. 1. The role's reach is a list Our auditor role gets five rules, and every one grants read only: { roleCode: UserRole.AUDITOR, subject: 'AuditLog', actions: ['read'], ownerField: null }, { roleCode: UserRole.AUDITOR, subject: 'OperationAudit', actions: ['read'], ownerField: null }, { roleCode: UserRole.AUDITOR, subject: 'GovernancePolicy', actions: ['read'], ownerField: null }, { roleCode: UserRole.AUDITOR, subject: 'ToolInventory', actions: ['read'], ownerField: null }, { roleCode: UserRole.AUDITOR, subject: 'ApprovalQueue', actions: ['read'], ownerField: null }, That list is everything the role can touch. Everything else is denied by the policy guard by default. It deliberately gets none of the USER ownership rules. This role is a reader of the governance record, not a user of the application. It should have no write action either. The day one of these rules grows a write, the role stops being an auditor. The shape of the rule is worth one sentence too. The underlying role seeds originally had only a subject and an owner field, and every seed implied manage. We added an optional actions; omitting it still means manage, so no existing role changed behaviour. Writing it is how a role narrows itself. Read access is written down, not inferred from management. 1. Fields: rebuilt from a whitelist, not deleted from a full row The role decides which surface you can reach. The field list decides what you can see on that surface. We keep one whitelist and rebuild the projection from it: only keys named on the list are written a column added tomorrow stays invisible to an audit reader until someone deliberately adds it to the list deleting forbidden fields from a full row would fail open: the next column leaks by default That is not a style choice. It is a choice about defaults. Deleting from a full row defaults to visible; rebuilding from a whitelist defaults to invisible. The fields left off the list have reasons. A conversation id is a pointer into a person's conversation. IP and device ids bind activity to a device or a person. A user's feedback about the assistant is not the audit record. Delegation context is free text that may quote a prompt. The bounded business intent and the agent and caller ids carry the attribution a reader actually needs. One test injects a key that is not on the list at runtime and asserts it does not appear in the projection. It is not testing deletion. It is testing the default. 1. Rows: organisation from the account, period from the request Beyond the role and the fields there is the question of which rows. The same surface, with the same fields, must still show different rows to different readers. The organisation comes from the account, not the request. An organisation parameter on the request can only narrow further, ANDed with the account scope, never replace it. An auditor who belongs to no organisation falls back to their own rows. A failed organisation lookup also falls back to their own rows. A failed lookup is not a reason to widen. The period comes from the request window. There is no per-reader period attribute today and no read-plane policy. Inventing one here would invent a dimension, so we use the window the caller asks for. Fetching a single row by id returns 404 when it is out of scope, the same answer as "no such row". The gate already passed; telling the reader that another organisation has that row is itself a disclosure. The by-id path and the list call the same scope function. Two implementations would eventually disagree. 1. Aggregates need the scope too, and so does the cache key Narrowing the list while leaving the counts narrow-free is the same as not doing it. The auditor's stats, cost, report and coverage statement all read through the same organisation scope. One thing is easy to miss: the aggregates are cached for 60 seconds, and the cache key has to carry the scope. We hit that class of defect. With no scope in the key, the first reader's organisation answer was served to the next reader for a minute. The fix is not to clear the cache. The fix is to put the scope in the key: all, org: or own:. It is the same class as a verification verdict computed for one database being served for another. A cache does not know who is asking, so the key has to. One thing we do not slice by organisation: the hash chain inside a signed evidence package. Slicing a chain and verifying the slice produces a false conclusion, so the chain keeps its full scope. Rows can be narrowed. A chain cannot. 1. What stays closed Listing what the auditor can reach matters as much as listing what they still cannot: - Audit log, single-event explanation, chain, verify, stats, cost, report: the auditor reads projected rows within their organisation scope; an admin reads full rows. - Operation audit: the auditor reads, but row scope is not yet narrowed by organisation; an admin reads. - Governance policy, its history, tool inventory, approval queue: the auditor reads; an admin reads, writes and decides. - Signed evidence package and verify chain payload: the auditor reads them unprojected so they stay verifiable; an admin reads them too. - Users, organisation, sessions, platform analytics, writes, approvals, execution: unreachable for the auditor; reachable for an admin. Three points deserve their own paragraph: The signed evidence package and the verify chain payload are not projected. They are verification artifacts whose value is the complete canonical payload. Projecting them makes them unverifiable. Privacy projection covers row-shaped responses, not the evidence itself. The auditor can read the approval queue but cannot decide it. Approving, rejecting and retrying execution stay admin-only. Reading the queue is not handling it. The same applies to flow approvers: the role list deliberately excludes the auditor. An auditor must not become the kind of person they are reviewing. Operation audit's read access is open, but its row scope is not narrowed yet. The service filters only by the request's userId; it does not apply the reader's account scope. This is an open item from the role's widening, and saying so here keeps the five rules from implying that every surface is already scoped. 1. The scene: widening one surface at a time Our first cut changed only the audit controller to read AuditLog. Operation audit still judged manage all, so the auditor reached that surface and got a 403. That was not an omission. It was deliberate: open one surface, see whether it is enough, then widen one surface at a time. Later, operation audit, governance policy, tool inventory and the approval queue each got their own read subject. Writes, approvals and execution never entered the widening. The order has a second benefit. Each surface can be verified on its own: what should be visible is visible, what should be blocked is blocked. Opening everything at once makes a hole impossible to attribute. 1. Five checks you can run Which surfaces can your read-only role reach? Can you enumerate them, or is it "roughly like admin"? Are new fields visible by default? Deleting from a full row fails open. Rebuilding from a whitelist fails closed. Does row scope come from the account or the request? Can the request only narrow it? Where does a missing organisation fall back to? Do the aggregates and the cache carry the same scope? A narrowed list with unscoped counts is not narrowed. Is the signed evidence projected? If it is, can it still be verified? Closing Back to the opening: giving someone the evidence is not giving them the system. What we do: five read-only rules define the role's entire reach; fields are rebuilt from a whitelist and new columns are invisible by default; organisation comes from the account and the period from the request; an out-of-scope row returns 404; aggregates and cache keys carry the same scope; signed evidence stays whole. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. Repository: rain6fish/KeelBase Four things stated plainly. The role is widened one surface at a time: everything outside the list is still unreachable, and that is default deny rather than unfinished work. Signed evidence cannot be projected, so privacy minimisation has an explicit boundary here. An auditor cannot decide approvals or serve as a flow approver. If they could, they would become the person they are reviewing. And operation audit's row scope is not narrowed by organisation yet: the read access is open, the scope is not, and that is an open item rather than a finished surface. Read-only is not "a little less permission". It is a reach you can enumerate. Drafted with an AI assistant. The system, the positions and the mistakes are mine.

One Timeout Cannot Answer Two Questions: Why an AI Write's Confirmation Needs Two Windows An AI asks to write something, and a person has to approve it. There are two kinds of "expired" in that sentence, and they are not the same thing: - **should the conversation keep waiting** for the approval, or move on without it - **can this row still be decided at all**, or is it dead We used one timeout to answer both. What that produced was a list showing "waiting on you" for a row where clicking was guaranteed to fail. This piece is about splitting those into two windows with one function each, and about the single point where a decision has to be arbitrated. ## 1. Why they look like one question Both read as "how long does a confirmation last", and once there is one number, the instinct is one column. But they answer different questions, and different things consume the answers: | The question | Who consumes it | What a wrong answer costs | |---|---|---| | should the conversation keep waiting? | the one stream that is waiting | waiting a little too long, or moving on a little early | | can this row still be decided? | the data layer and every decision entry | **a click that cannot execute**, or a row that is dead and still actionable | A wrong answer to the first costs **experience**. A wrong answer to the second costs **consistency**. And when they share a value, the second question's answer gets dragged around by the first one's needs. ## 2. The scene: computed in two places, held together by a comment This happened to us. The offline window's boundary was computed **twice**: the read path compared timestamps **in memory**, and the maintenance sweep compared them **in SQL**. What kept the two in agreement was **a comment in the code**, saying roughly that the predicate here matches the one over there. **A comment is not a predicate.** The day the two disagree, you get a row like this: > the list shows "waiting on you", while the sweep has already expired it. So the click fails, necessarily. And it is not an occasional failure: **the list manufactures it**, by presenting a dead row as a pending task. What makes this hard to find is that it looks like "the click sometimes fails", while the cause is two places holding different definitions of the same instant. ## 3. Fix one: two windows, and one function per boundary There are two halves to the fix. **First, admit they are two windows.** - the **in-conversation wait** is short, 60 seconds by default, and its expiry only **ends that wait** so the conversation can continue. It does not touch the data. - the **offline window** is long, 24 hours by default, and the row stays decidable inside it. Only its expiry flips the row to timed out. **Second, every boundary is computed by exactly one function.** The read path and the sweep both call it and get the same instant. So "shown as actionable" and "already expired" become structurally impossible to be true at once. The point is not that the arithmetic is right. The point is that **there is one place doing it**. Two places and all you have left is a comment promising they agree. ## 4. Fix two: one arbitration point for decisions Once there are two windows, there are **two decision entries**: the button in the conversation, and the pending list afterwards. They can race. So the second thing is to make them contend for the same thing. What we do is a **conditional update in the database**, where the condition includes "the current status is still pending". Then: - the update touches one row, so this decision stands and it executes - the update touches zero rows, so somebody already decided, and this one does not execute **Why not read, then decide?** Because there is a gap between the read and the write, and both decisions can read "still pending". That is a decision placed before the action without the arbitration being placed before the action. A conditional update is atomic: whoever changes the row first owns the decision. Two entries, one arbitration point. That is the whole of this section. ## 5. The second scene: one "we don't know" read as "yes" Another one of ours. A decision has to be written before it counts. **What if that write throws?** The old behavior swallowed the error and released the in-memory wait anyway. Which gave this chain: 1. the database hiccups, the write fails 2. the in-memory wait is released, so the tool runs 3. the row is still pending 4. the next decision, in the conversation or from the pending list, sees "still pending" and runs it a second time **A "we do not know whether the decision landed" was read as a "yes".** The fix is plain: without evidence that the decision reached the database, do not release. Fail this one and let the person retry, rather than translating uncertainty into execution. One thing worth being precise about: for external tools with no idempotency key, that second run is a real replay and the target really does receive it twice. On that path, failing closed is not caution. It is required. ## 6. A boundary this design does not cross **The arbitration point guarantees at most one decision.** It does not guarantee the tool runs at most once. If the same request gets **resent**, by a model retry or a restart, that is a different problem and it has to be solved at a different layer. The two must be kept apart: one is "the same decision cannot be taken twice", the other is "the same call cannot be sent twice". Rolling them into a single sentence about "we guarantee it executes once" is overclaiming. We put that in the result messages and in the docs rather than leaving it to be discovered. ## 7. A check you can run yourself Four questions. 1. **How many questions does your timeout answer?** If one value decides both "does the UI keep waiting" and "can this row still be used", that is a shared value. 2. **How many implementations does the boundary have?** Two is one comment away from drifting. 3. **What arbitrates a racing decision?** An in-memory check is not arbitration. A conditional update, or any atomic compare-and-swap, is. 4. **What is the default when a decision fails to persist?** If the next step after a failed write is "execute anyway", the default is allow. ## Closing Back to the opening: **one timeout cannot answer two questions.** What we do: split it into two windows, 60 seconds of in-conversation waiting and 24 hours of decidable pending; compute each boundary in exactly one function; make every decision contend for one conditional update; and release nothing without evidence that the decision landed. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system and re-checks identity, authorization, scope, risk and confirmation state before a tool actually runs. Repository (a Gitee mirror is the faster route from inside China): - GitHub: rain6fish/KeelBase - Gitee: gitee.com Two things stated plainly. The boundary above: we guarantee at most one decision, not at most one execution. And a premise of the whole design. It assumes decisions can be persisted. If the database is unavailable we refuse rather than allow, which for some systems means confirmation stops working entirely. That is deliberate. If confirmation means "nothing runs that was not confirmed", then when nothing can be confirmed, nothing should run. --- *Drafted with an AI assistant. The system, the positions and the mistakes are mine.*

AI Execution Governance, Part 7: One Timeout Cannot Answer Two Questions An AI asks to write something, and a person has to approve it. There are two kinds of "expired" in that sentence, and they are not the same thing: - should the conversation keep waiting for the approval, or move on without it - can this row still be decided at all, or is it dead We used one timeout to answer both. What that produced was a list showing "waiting on you" for a row where clicking was guaranteed to fail. This piece is about splitting those into two windows with one function each, and about the single point where a decision has to be arbitrated. 1. Why they look like one question Both read as "how long does a confirmation last", and once there is one number, the instinct is one column. But they answer different questions, and different things consume the answers: | The question | Who consumes it | What a wrong answer costs | |---|---|---| | should the conversation keep waiting? | the one stream that is waiting | waiting a little too long, or moving on a little early | | can this row still be decided? | the data layer and every decision entry | a click that cannot execute, or a row that is dead and still actionable | A wrong answer to the first costs experience. A wrong answer to the second costs consistency. And when they share a value, the second question's answer gets dragged around by the first one's needs. 2. The scene: computed in two places, held together by a comment This happened to us. The offline window's boundary was computed twice: the read path compared timestamps in memory, and the maintenance sweep compared them in SQL. What kept the two in agreement was a comment in the code, saying roughly that the predicate here matches the one over there. A comment is not a predicate. The day the two disagree, you get a row like this: > the list shows "waiting on you", while the sweep has already expired it. So the click fails, necessarily. And it is not an occasional failure: the list manufactures it, by presenting a dead row as a pending task. What makes this hard to find is that it looks like "the click sometimes fails", while the cause is two places holding different definitions of the same instant. 3. Fix one: two windows, and one function per boundary There are two halves to the fix. First, admit they are two windows. - the in-conversation wait is short, 60 seconds by default, and its expiry only ends that wait so the conversation can continue. It does not touch the data. - the offline window is long, 24 hours by default, and the row stays decidable inside it. Only its expiry flips the row to timed out. Second, every boundary is computed by exactly one function. The read path and the sweep both call it and get the same instant. So "shown as actionable" and "already expired" become structurally impossible to be true at once. The point is not that the arithmetic is right. The point is that there is one place doing it. Two places and all you have left is a comment promising they agree. 4. Fix two: one arbitration point for decisions Once there are two windows, there are two decision entries: the button in the conversation, and the pending list afterwards. They can race. So the second thing is to make them contend for the same thing. What we do is a conditional update in the database, where the condition includes "the current status is still pending". Then: - the update touches one row, so this decision stands and it executes - the update touches zero rows, so somebody already decided, and this one does not execute Why not read, then decide? Because there is a gap between the read and the write, and both decisions can read "still pending". That is a decision placed before the action without the arbitration being placed before the action. A conditional update is atomic: whoever changes the row first owns the decision. Two entries, one arbitration point. That is the whole of this section. 5. The second scene: one "we don't know" read as "yes" Another one of ours. A decision has to be written before it counts. What if that write throws? The old behavior swallowed the error and released the in-memory wait anyway. Which gave this chain: 1. the database hiccups, the write fails 2. the in-memory wait is released, so the tool runs 3. the row is still pending 4. the next decision, in the conversation or from the pending list, sees "still pending" and runs it a second time A "we do not know whether the decision landed" was read as a "yes". The fix is plain: without evidence that the decision reached the database, do not release. Fail this one and let the person retry, rather than translating uncertainty into execution. One thing worth being precise about: for external tools with no idempotency key, that second run is a real replay and the target really does receive it twice. On that path, failing closed is not caution. It is required. 6. A boundary this design does not cross The arbitration point guarantees at most one decision. It does not guarantee the tool runs at most once. If the same request gets resent, by a model retry or a restart, that is a different problem and it has to be solved at a different layer. The two must be kept apart: one is "the same decision cannot be taken twice", the other is "the same call cannot be sent twice". Rolling them into a single sentence about "we guarantee it executes once" is overclaiming. We put that in the result messages and in the docs rather than leaving it to be discovered. 7. A check you can run yourself Four questions. 1. How many questions does your timeout answer? If one value decides both "does the UI keep waiting" and "can this row still be used", that is a shared value. 2. How many implementations does the boundary have? Two is one comment away from drifting. 3. What arbitrates a racing decision? An in-memory check is not arbitration. A conditional update, or any atomic compare-and-swap, is. 4. What is the default when a decision fails to persist? If the next step after a failed write is "execute anyway", the default is allow. Closing Back to the opening: one timeout cannot answer two questions. What we do: split it into two windows, 60 seconds of in-conversation waiting and 24 hours of decidable pending; compute each boundary in exactly one function; make every decision contend for one conditional update; and release nothing without evidence that the decision landed. The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system and re-checks identity, authorization, scope, risk and confirmation state before a tool actually runs. Repository (a Gitee mirror is the faster route from inside China): - GitHub: rain6fish/KeelBase - Gitee: gitee.com Two things stated plainly. The boundary above: we guarantee at most one decision, not at most one execution. And a premise of the whole design. It assumes decisions can be persisted. If the database is unavailable we refuse rather than allow, which for some systems means confirmation stops working entirely. That is deliberate. If confirmation means "nothing runs that was not confirmed", then when nothing can be confirmed, nothing should run. --- Drafted with an AI assistant. The system, the positions and the mistakes are mine.