KetanShukla.dev
API5 min read

How to unit-test an iteration cap without ever calling a model

The agent loop's most important safety property is that it stops. Testing that against a live API is slow, expensive and flaky — so I put one interface between the loop and the execution engine, and the test became four lines.

An agent loop's most important property is not that it produces good answers. It is that it stops.

A loop that never terminates does not fail loudly — it fails as a bill. The model keeps requesting tools, the loop keeps obliging, and the only signal is a number on an invoice at the end of the month.

So the iteration cap deserves a test. And the moment you try to write one, you hit the actual problem: the loop is wired into a durable-execution engine and a model API, and neither of those belongs in a unit test.

The dependency that makes it hard

In production, each turn of my loop runs inside an Inngest step.run(...) — a durable checkpoint, so a failed step can retry without redoing the whole run. That is exactly what you want in production and exactly what you cannot have in a test: it needs a running Inngest dev server, real event delivery, and it makes a millisecond assertion into a multi-second integration test.

The instinct is to reach for a mocking library and stub the runtime. That produces a test which passes when your mock is right and says nothing about whether the loop is right.

One interface, defined by what the loop actually needs

The loop does not need Inngest. It needs something that runs a named function and returns its result. That is thirteen lines of type:

src/lib/agentLoop.ts
// A minimal shape compatible with Inngest's `step.run(id, fn)`, so this loop
// can be driven by the real Inngest step tools in production and by a plain
// pass-through implementation in tests (no checkpointing needed there).
export interface StepLike {
  run<T>(id: string, fn: () => Promise<T>): Promise<T>;
}

Everything else the loop touches gets the same treatment — the model call and the tool executor become injected functions rather than imported clients:

src/lib/agentLoop.ts
export interface RunAgentIterationsDeps {
  step: StepLike;
  /** Calls the model with the running message history and returns its response. */
  createMessage: (messages: MessageParam[]) => Promise<Message>;
  /** Executes a single tool call. Never throws; catch internally. */
  executeTool: (
    name: string,
    input: Record<string, unknown>,
  ) => Promise<{ ok: boolean; output: string }>;
  maxIterations?: number;
}

Note what StepLike is not: it is not an abstraction over durable execution. It does not have sleep, or waitForEvent, or retry configuration. It is precisely the one method this loop calls, and nothing else.

The test implementation of the runtime is two lines

src/lib/agentLoop.test.ts
// A pass-through "step" that just runs the function immediately, with no
// checkpointing — good enough to exercise the loop logic in isolation.
const passThroughStep: StepLike = {
  run: (_id, fn) => fn(),
};

That is the whole substitute for the durable-execution engine. It preserves the only semantics the loop depends on — call this and give me the result — and discards the ones it does not care about.

And now the test says exactly what it means

src/lib/agentLoop.test.ts
it("stops after MAX_ITERATIONS when the model always requests a tool call", async () => {
  let callCount = 0;
  const createMessage = vi.fn(async () => {
    callCount++;
    return makeToolUseMessage(`tool_${callCount}`);
  });
  const executeTool = vi.fn(async () => ({ ok: true, output: "some result" }));
 
  const result = await runAgentIterations([{ role: "user", content: "task" }], {
    step: passThroughStep,
    createMessage,
    executeTool,
  });
 
  expect(createMessage).toHaveBeenCalledTimes(MAX_ITERATIONS);
  expect(result.iterations).toBe(MAX_ITERATIONS);
  expect(result.finalAnswer).toBe(MAX_ITERATIONS_FALLBACK_MESSAGE);
});

A model client that always asks for another tool call. That is a pathological model — one that never terminates — and it is the exact adversary the cap exists to defend against. You cannot reliably produce that behaviour from a real model, and you certainly cannot produce it on demand, in CI, in milliseconds, for free.

The companion test proves the other direction:

src/lib/agentLoop.test.ts
it("stops early once the model returns a final text answer", async () => {
  const createMessage = vi
    .fn()
    .mockResolvedValueOnce(makeToolUseMessage("tool_1"))
    .mockResolvedValueOnce(makeFinalMessage("Here is the answer."));
  // ...
  expect(result.iterations).toBe(2);
  expect(result.finalAnswer).toBe("Here is the answer.");
});

Together they pin both boundaries: the loop stops when the model is done, and stops anyway when the model is not.

What the fallback message buys

Look at what happens on the cap:

export const MAX_ITERATIONS_FALLBACK_MESSAGE =
  "(reached maximum iterations without a final answer; see step trace for details)";

Not an exception. Not an empty string. A run that completed, with an honest final answer explaining why it is unsatisfying.

This matters more than it looks. If hitting the cap throws, the run is marked failed, the persisted steps look like a crash, and the user gets an error page for something that is not an error — the system worked exactly as designed. If hitting the cap returns an empty answer, the run looks successful and is silently useless.

An explicit terminal message means the trace is still readable, the run status is accurate, and the user can see the six things the agent tried before running out of room.

The generalisation

The mocking discussion usually gets framed as "mocks versus integration tests", which is the wrong axis. The useful question is: which of this component's dependencies carry semantics I am testing, and which are infrastructure I am tolerating?

  • The model's behaviour is what the loop reacts to → I need to control it precisely, so it is injected and I supply pathological cases.
  • Durable checkpointing is infrastructure the loop happens to run inside → I need it to not be there, so it is an interface with a trivial implementation.

Once those are separated, the loop becomes a pure function of its inputs, and the safety property that actually costs money if it breaks gets a test that runs on every push in under a second.

Both files are worth reading side by side: src/lib/agentLoop.ts and src/lib/agentLoop.test.ts.

testingvitestagent-loopdurable-executioninngesttypescript

Read next