Field notes · 15 posts
Building agentic systems that can be trusted
Everything here comes out of a deployed project — the protocol decisions, the failures, and the numbers from real runs. Four subjects, and nothing else.
Model Context Protocol
Servers, hosts, tools, resources, prompts, sampling — and the trust boundaries that decide who is allowed to do what.
Retrieval-Augmented Generation
Chunking, embeddings, similarity, citations, and the failure modes that look like model problems but are retrieval problems.
Agent-to-Agent
Agents that delegate to other agents: sub-agent spawning, approval queues that span a tree of workers, and the orchestration lessons that predate LLMs.
Model APIs & Application Surfaces
Tool use, structured output, prompt caching, durable execution, and the route handlers that sit between a model and a production system.
Latest
Hybrid retrieval made my results worse
Everyone recommends fusing lexical and dense retrieval, then reranking. I measured it on a hand-labelled set and the default recommendation finished third of six configurations — behind dense retrieval on its own.
- A2A6 min
Accuracy is the wrong metric for an agent router
A keyword baseline scored 0.625 accuracy and claimed every single out-of-scope task. Its abstention rate was zero — and accuracy hid that completely. Three routers measured across A2A capability cards.
- MCP6 min
A stateless MCP tool re-embeds its corpus on every call
The documents arrive as an argument, so the obvious implementation encodes all of them again each time the agent asks a question. That was 1,040 ms per search. A cache keyed on the exact string took it to 1.2 ms.
- API5 min
Nightly builds for a national retail chain, and why I checkpoint agent runs
Packaging POS releases where a failed rollout stopped checkout and store revenue, with a rollback path per store. The instinct that made durable execution obvious twenty years later.
- API6 min
A multi-minute agent run should not die on one transient failure
The v1 research agent lost entire runs to a single timeout. Rebuilding it on durable execution meant every unit of work checkpoints independently — and the interesting part was handling the failures that used to disappear silently.
- API5 min
How to unit-test an iteration cap without ever calling a model
The agent loop's most important safety property is that it stops. Testing that against a live API is slow, expensive and flaky — so I put one interface between the loop and the execution engine, and the test became four lines.
- API6 min
One JSON schema, three formats, no branching logic
A tabular invoice, an ASCII-art grocery receipt, and a prose billing email through one unchanging code path — and the null-safe money handling that decides whether the result is usable in a financial system.
- RAG6 min
Row-level security is a schema decision, not a code-review convention
Four tables, four policies, and one keyword in a Postgres function that decides whether any of it works. Verified against a second account rather than assumed — because a tenancy bug is not the kind you find by reading.
- RAG6 min
Deleting the similarity threshold is what made retrieval work
A fixed cosine cutoff was silently discarding the chunks that held the answer. Removing it entirely — and letting the prompt judge relevance instead — fixed a bug I had spent a week blaming on the model.
- A2A5 min
I built message orchestration for a federal immigration system. It was the same problem.
BizTalk pipelines, maps and orchestrations against XSD schemas, versus an agent loop routing tool calls with validation and failure handling. Twenty years apart, one problem — and the older field already solved three things the newer one keeps rediscovering.
- MCP7 min
8 allowed at 13.0¢. 5 refused at $0.00.
MCP sampling lets a server spend your money using your API key. A gate that reasons about estimated cost before the call means a refusal is free — and that changes what you gate on.
- A2A7 min
One agent found 0 of 36. A crew found 36 of 36, for half the tokens.
A 240-item workload measured both ways. The single agent does not fail because it does more work — it fails because it re-sends everything it has already read on every iteration. Including the part where my first measurement was wrong.
- MCP7 min
A server-supplied danger hint is not a permission model
MCP lets a server declare its own tools destructive. If your approval gate reads that flag, any server can walk through your gate by lying. The fix is thirty lines and it lives entirely on the host.
- API5 min
One cache breakpoint cut my agent loop's input cost by 47%
An agent loop re-sends its entire prefix on every iteration. Real token counts before and after a single cache_control marker — plus the two ways to silently break it and never see an error.
- MCP5 min
An MCP server is one Next.js route handler
No framework, no scaffolding, no SDK ceremony. Four tools, a resource, and a prompt in a single file — and the one line in it that decides whether your tool ever gets used.