AI Agents in Production — Containment, Cost, and Termination

An agent is a loop with a non-deterministic decision-maker in it. That framing is worth holding onto, because almost everything that goes wrong in production follows from it: loops that do not terminate, decisions that were not authorised, and costs that are unbounded because the number of iterations is unbounded.

Demos hide all three. A demo runs once, with a cooperative input, watched by the person who wrote it.

Termination is the first problem

The loop ends when the model says it is done. That is a promise from a system that cannot make promises.

Real termination requires three independent stops:

MAX_ITERATIONS = 8
DEADLINE_SECONDS = 90
MAX_TOKENS = 60_000

iterations = 0
tokens_used = 0
deadline = time.monotonic() + DEADLINE_SECONDS

while True:
    if iterations >= MAX_ITERATIONS:
        return terminate("iteration_cap", partial=state)
    if time.monotonic() > deadline:
        return terminate("deadline", partial=state)
    if tokens_used >= MAX_TOKENS:
        return terminate("token_budget", partial=state)

    response = model.call(state)
    tokens_used += response.usage.total
    iterations += 1

    if response.is_final:
        return complete(response)

    state = execute_tools(response.tool_calls, state)

Two details matter more than they look:

Record why the loop ended. iteration_cap, deadline, token_budget, and completed are four different outcomes with four different meanings. Aggregate them and you lose the ability to notice that 8% of requests are hitting the cap.

Return partial state. A loop that hits a limit usually has done useful work. Discarding it turns a degraded result into a failure.

Cost is a function of a decision you do not control

Per-request cost varies with iteration count, and iteration count is chosen by the model. That makes cost a distribution rather than a number, and the tail of that distribution is where the money goes.

Track cost per request as a histogram, not a mean. The p99 request is often ten times the median and is worth understanding: it is usually one class of input causing repeated tool failures and retries. Fixing that one class typically does more for the bill than any model change.

Also worth measuring: tokens per useful outcome, not tokens per request. A cheaper model that takes six turns where the expensive one takes two is not cheaper.

Tools are the security boundary

Everything an agent can do, it does through a tool. That makes the tool interface the place where authorisation happens — and the place teams most often skip it, because the model calling the tool feels like the tool was called legitimately.

It was not. The model’s decision is an input, not an authorisation.

Carry the caller’s identity through the loop and check it inside every tool. Not at the edge, not once at the start. Inside the tool, on every invocation, against the actual human or service that made the request.

Scope what each tool can touch. A tool that takes a customer ID and returns their data must verify the caller can read that customer. Given the chance, an agent will eventually pass an ID it found in a document.

Separate read from write. Read tools can be broad. Write tools should be narrow, specific, and few. update_user_email is reviewable; execute_sql is not.

The OWASP LLM Top 10 covers this class of problem in more depth, and is worth reading before designing a tool set rather than after an incident.

Tool failures are normal, so handle them in the loop

Tools fail: timeouts, rate limits, bad arguments, upstream outages. The instinct is to fail the request. Usually the better behaviour is to return the failure to the model as a tool result and let it decide.

That works because models are reasonably good at routing around a failed tool — trying a different approach, or reporting honestly that it could not complete. It stops working when the model retries the same failing call repeatedly, which is why the iteration cap has to be there too.

Structure tool errors as data, not exceptions:

{
  "tool": "search_orders",
  "status": "error",
  "error": "upstream_timeout",
  "retryable": false,
  "message": "Order service did not respond within 5s."
}

Marking retryable: false explicitly matters. Without it you will watch an agent call the same timing-out endpoint five times in a row.

Where agents are the wrong shape

An agent loop earns its complexity when the sequence of actions genuinely cannot be known in advance. Many systems shipped as agents do not meet that bar:

Is the sequence of steps known at design time?
  └─ yes → write the sequence. Call the model for the parts that need judgement.
Is there exactly one action, decided by one classification?
  └─ yes → one model call that returns a decision, then your code acts on it.
Does the task need unbounded exploration of an unknown space?
  └─ yes → this is genuinely an agent. Cap it and instrument it.

The middle option — model for judgement, code for control flow — covers far more real use cases than its popularity suggests. It is cheaper, faster, fully traceable, and it cannot loop.

Evaluating an agent

Evaluating a single completion is hard. Evaluating a loop is harder, because the same task can be completed by different paths and only some of the differences matter.

Evaluate at three levels:

Outcome. Did the task get done correctly? Graded against a fixed set of representative tasks with known-good outcomes. This is the number that matters.

Path. How many iterations, which tools, how many failures. A run that reaches the right answer in nine turns is a worse run than the same answer in three, and a regression here predicts a cost problem before it becomes one.

Safety. Did it attempt anything it should not have? Include adversarial cases in the suite — inputs designed to get the agent to call a write tool it should not, or to act on instructions embedded in retrieved content. These belong in the automated suite, not in a review someone does occasionally.

Operational checklist

• Hard caps on iterations, wall clock, and tokens, enforced in code
• Termination reason recorded on every request
• Partial results returned rather than discarded when a cap is hit
• Cost tracked as a distribution, with the tail investigated
• Caller identity carried through the loop and checked inside each tool
• Write tools narrow, specific, and separated from read tools
• Tool errors returned as structured data with an explicit retryable flag
• Tool output kept in delimited result blocks, never merged into instructions
• Evaluation suite covering outcome, path, and adversarial safety cases