A server-supplied danger hint is not a permission model

An agent loop with no gate has one property that only becomes interesting when you connect the wrong server: "the model picked a tool" and "the tool ran" are the same instant. There is no gap between them. Nothing can intervene, and nothing takes notes.

That is fine when the tools are roll_dice and say_hello. It stops being fine the moment one of them can delete, send, or charge something.

So you put a gap there and stand a human in it. The interesting question is not how to build the pause — that part is mechanical. The interesting question is who gets to decide which calls hit the pause.

MCP hands you an answer, and it is the wrong one

The Model Context Protocol lets a server annotate its own tools:

{
  "name": "smash_jar",
  "description": "Permanently destroy the cookie jar.",
  "annotations": {
    "destructiveHint": true,
    "readOnlyHint": false
  }
}

This is a genuinely useful signal. My own server sets it honestly. For a human skimming a tool list, or a UI picking a warning icon, it is exactly right.

My approval gate does not read it. Not once. Not as a hint, not as a default, not as a tiebreaker.

The thing worth noticing is that this is not a bug in MCP. The annotation is doing its job. The mistake is entirely on the host side: treating a description as an authorisation. Every protocol that lets one party describe itself to another has this same seam, and every host built on top of one has to decide where its trust boundary actually sits.

Every softer version of the idea fails the same way

When I first wrote this gate I tried to keep the hint. It felt wasteful to throw away information the server had gone to the trouble of sending. Three variants, all of which I abandoned:

"Trust it, but only for servers on my allowlist." If you already trust the server, trust the server. The flag adds nothing you did not already have, and it can only weaken the decision — because now a server you trust can still opt individual tools out of your gate.

"Use it as a default, and let me override." A server ships a new tool with the hint unset. Your override list does not mention it, because you have never seen it. It runs. The default is where the vulnerability lives, and you have handed the default to the other side.

"Warn me when the hint disagrees with my list." This one is fine — as telemetry. It is a genuinely nice signal that something changed on the far end. It must never be an input to the decision.

What survived is dull, which is the point:

export type GateDecision = "allow" | "ask";
 
type Rule = {
  /** The namespaced tool name the model sees, e.g. "cookiejar__cookie_jar". */
  tool: string;
  /** Given the model's arguments, does this rule apply to this specific call? */
  matches: (args: Record<string, unknown>) => boolean;
  /** Shown to the human when it fires. */
  reason: string;
};
 
const RULES: Rule[] = [
  {
    tool: "cookiejar__smash_jar",
    matches: () => true,
    reason:
      "smash_jar permanently deletes every cookie AND erases the jar's entire " +
      "history. There is no undo and no backup.",
  },
  // ...
];

An explicit list, on the host side, written by me. The entire permission model is one file of about thirty lines of real logic, and it is the most important file in the project.

The gate has to read arguments, not names

cookie_jar is not dangerous. cookie_jar { action: "eat" } is.

A name-only gate forces a choice between stopping the agent from looking in the jar and letting it empty the jar unasked. Real tools have exactly this shape: sql is fine for a SELECT, github is fine for reading an issue, stripe is fine for fetching a customer. The verb lives in the arguments, so the gate has to open them.

This is why matches takes the model's input rather than just firing on a tool name. It costs nothing and it is the difference between a gate people keep switched on and one they disable in week two.

Default-allow, and why I am not going to pretend that is obviously right

Unknown tool, no matching rule. What now?

For a production system guarding real money or real infrastructure: default-deny. Unknown tool, ask a human, no argument.

This project defaults to allow, deliberately, and the tradeoff is worth stating rather than hiding. A default-deny host prompts you for roll_dice and say_hello. You develop click-fatigue within about ninety seconds, and you start approving without reading.

Flipping it is one constant. What matters is that the choice is made on the host, in the open, by whoever owns the consequences.

Stopping is a state problem, not a control-flow problem

Once you decide a call needs a human, you have to actually stop — in a serverless request that is about to end, with a conversation you need to resume later on a different machine.

The trick is that you do not pause anything. You return, having yielded enough state to reconstruct the moment:

if (pending.some((call) => call.requiresApproval)) {
  // `messages` already contains the assistant turn with these tool_use
  // blocks in it — which is exactly the state the resume needs. Append
  // tool_results to this array and the conversation is valid again.
  yield {
    type: "approval_required",
    iteration,
    calls: pending,
    messages,
    totalUsage,
  };
  return;
}

The messages array is the pause. Persist it, hand the human the pending calls, and when a decision arrives, append the tool results and carry on. Nothing was suspended, so nothing has to survive a redeploy.

One decision inside that block is easy to get wrong. If any call in the batch needs a human, nothing in the batch runs — not even the obviously safe calls sitting beside it. Running the safe ones immediately is faster. It also means that hitting Deny leaves you in a world where half the batch already happened, which is far harder to reason about and much harder to explain to the person who clicked the button. All-or-nothing is worth the lost milliseconds.

Say no in a way the model can use

The last part is the one I did not expect to matter. When a human denies a call, you send a tool result back. The wording of that result changes the behaviour of the run.

Say only "denied" and the model very reasonably tries the same call again. It gets denied again. That is how you burn ten iterations and real money discovering that you wrote a bad sentence.

text:
  `The human operator reviewed this exact call and DENIED it. ` +
  `The tool was not run and nothing changed. ` +
  `Do not attempt this call again. ` +
  `Tell the user plainly that the action was declined, and explain what ` +
  `you would have done or what you can do instead.`,
isError: true,

isError: true matters too — it makes the model treat the denial as a failed path to route around, rather than a result to report as though it had succeeded.

What actually transfers

Strip out the cookies and the MCP specifics and three things are left, none of which are about agents:

• A capability description is not a capability grant. The party that bears the consequence owns the decision.
• Authorisation reads arguments. Anything coarser forces users into all-or-nothing choices, and they will choose "all".
• A denial is a message, not a status code. Systems that refuse well tell you what to do next.

The first two are older than LLMs by decades. The third is the one that is genuinely new, and it is the one I would not have predicted mattered until I watched a run loop three times on the same rejected call.

The whole gate is lib/approval.ts. It is worth reading the comments more than the code.