Designing Reliable AI Agents

Most "AI agent" demos work because someone is watching them. You give the model a goal, it reasons out loud, calls a few tools, and produces something plausible. That is a prototype, not a system. The moment an agent runs unattended, against real APIs, on inputs it has never seen, the interesting engineering begins — and almost none of it is about the prompt.

I've come to treat agents as distributed systems that happen to have a language model in the control path. Once you adopt that frame, the failure modes become familiar: unbounded state, non-idempotent side effects, missing backpressure, and no way to tell whether the thing is actually working. This post is how I design agents that survive contact with production.

The control loop is the product

Every agent is a loop: perceive the current state, plan a next step, act on the world, observe the result, repeat until done or dead. This is not a metaphor — it should be an explicit, readable function in your codebase, not something emergent inside a mega-prompt.

async def run(goal: str, ctx: Context, max_steps: int = 12) -> Result:
    for step in range(max_steps):
        obs = ctx.snapshot()                 # perceive: bounded view of state
        decision = await plan(goal, obs)     # plan: model picks one tool call
        if decision.done:
            return Result.ok(decision.answer)
        result = await act(decision.tool, decision.args, ctx)  # act
        ctx.record(decision, result)         # observe: append to bounded log
    return Result.exhausted(ctx)

The value of making the loop explicit is that every hard requirement now has an obvious home. Step limits go in the for. Idempotency lives in act. Context management lives in snapshot. When the loop is a diagram in your head instead of code, none of those controls have anywhere to attach, and you end up bolting them onto the prompt where they don't belong.

An agent is a control loop with a language model in it, not a language model that occasionally loops.

Why naive ReAct loops fail in production

The textbook ReAct pattern — interleave reasoning and acting, feed every observation back into an ever-growing transcript — is a great teaching tool and a poor production architecture. It fails in specific, predictable ways.

• Context grows without bound. Each tool result gets appended verbatim. Twenty steps in, you're paying to re-read a 40KB API response the model already summarized. Latency and cost climb linearly while relevance drops.
• Errors compound instead of resetting. A malformed tool call produces a confusing observation, which produces a worse plan, which produces a worse call. Without an explicit recovery path, the loop spirals.
• No idempotency. The model retries a failed step by simply calling the tool again. If that tool charged a card or sent an email, "retry" is now a bug with financial consequences.
• Termination is vibes-based. The loop ends when the model decides it's done. On a bad day it decides that at step 40, or never.

None of these are model-quality problems. A smarter model fails the same way, just later. The fix is architectural: constrain what the loop can do, not just what the model can say.

Tool interfaces are your real API surface

The tools are where the agent touches reality, so their design matters more than the prompt. I hold agent tools to the same standard as a public API, with a few extra rules that exist specifically because the caller is a probabilistic model.

Make each tool do one narrow thing with an unambiguous name. search_orders(customer_id, status) is a good tool. manage_orders(action, payload) is a trap — you've pushed a routing decision into free-form arguments the model will get wrong.

Return structured, compact results and put errors in-band as data the model can act on, not as exceptions that blow up the loop.

def refund_order(order_id: str, amount_cents: int, idempotency_key: str) -> ToolResult:
    if amount_cents <= 0:
        return ToolResult.error("amount_cents must be positive")  # recoverable, in-band
    ok = payments.refund(order_id, amount_cents, key=idempotency_key)
    return ToolResult.ok({"refunded": ok, "order_id": order_id})

Two details there are load-bearing. The idempotency_key means a retried call collapses into the same refund instead of a second one. The validation error returns as a ToolResult the model can read and correct, rather than throwing and forcing the loop to guess. Design the tool so the worst thing the model can do is get a clear error back.

Bound the state, curate the context

The single highest-leverage decision in an agent is what the model gets to see each step. Left alone, context is a garbage heap; treated as a curated working set, it becomes the thing that keeps the agent coherent.

I keep two separate things. There's durable state — the structured facts the agent has established, stored as typed data outside the transcript. And there's the context window — a bounded, freshly-rendered view assembled from that state for the current step. The window is derived, never accumulated.

def snapshot(self) -> Observation:
    return Observation(
        goal=self.goal,
        facts=self.store.top_k_relevant(self.goal, k=8),  # curated, not everything
        last_result=self.history[-1] if self.history else None,
        steps_remaining=self.max_steps - len(self.history),
    )

Because the window is rebuilt from state each turn, long-horizon tasks don't degrade as the transcript grows — the agent at step 30 sees a clean, relevant view, not thirty steps of accumulated noise. This also makes the agent resumable: persist the state, and you can crash and restart mid-task without losing the thread.

Guardrails and verification

Trust the model to make plans; don't trust it to be the last line of defense. Verification belongs in deterministic code around the loop, at three layers.

1. Input guardrails validate and constrain what enters the loop — scope, allowed tools, and any policy that must hold regardless of what the user asked.
2. Action guardrails sit in front of side effects. High-consequence tools (refunds, deletes, external sends) pass through a policy check, and the risky ones require an explicit approval step rather than firing autonomously.
3. Output verification checks the final result against the goal before returning it — schema validation, a checklist, or a cheap second model asked only "does this actually satisfy the request?"

The principle: the model proposes, deterministic code disposes. Anything that can cause irreversible harm should be gated by something you can read and test, not by a well-worded instruction.

Evaluate like you mean it

You cannot improve an agent you can't measure, and "it looked good in the demo" is not measurement. I build an eval harness before I trust an agent with anything real, and I treat it like a test suite that happens to be non-deterministic.

Start with a frozen set of tasks with known-good outcomes — not just inputs, but the state the world should be in afterward. Score task success, not token-level similarity: did the refund happen, was it the right amount, did it happen exactly once. Track steps to completion and tool-error rate as health signals; a rising step count usually means the context strategy is degrading before success rate visibly drops. And keep a regression set of every real failure you've seen, so fixing one thing doesn't quietly break another.

Run this on every prompt change, every tool change, every model upgrade. A model upgrade especially — a "better" model can be worse on your specific tool schema, and you only find out because the eval caught it.

Closing

Reliable agents aren't the product of a clever prompt; they're the product of boring, well-understood systems engineering applied to a new kind of component. Make the control loop explicit. Design tools like a real API with idempotency baked in. Curate context from bounded state instead of letting a transcript metastasize. Put guardrails and verification in deterministic code, and hold the whole thing to an eval suite you actually run.

Do that, and the model becomes the interesting, replaceable part — while the reliability lives in the architecture, exactly where you can reason about it.