The retry that ran all night

The invoice landed at month end, three times the usual size. Finance asked which project had grown. Nobody could say, because no project had — the number came off one line, a count of calls to a model, run by an agent that had not been looked at since the week it went live.

It had not strayed from its job. It had been told to watch a queue and raise a flag when an order looked wrong, and it had kept doing exactly that, including the part where it failed to reach a system that had been down for most of the night, and tried again every few minutes until morning. Nobody had told it to stop retrying. Nobody had told it when to stop, full stop, which is the part that should have annoyed them. It was still inside the job it was given when it ran up a bill nobody noticed until the bill arrived.

Why we build it this way

A scope is a good document. It says what an agent may read, what it may do, and where it has to stop and ask. Most of the care that goes into standing up an agent goes into writing that scope well, and it should.

But a scope describes correct behaviour. It says nothing about how many times, or at what cost. An agent can stay entirely inside its scope — reading only what it was told to read, doing only what it was told to do — and still cost a company a month's budget overnight, because the failure that actually costs money is rarely the agent doing the wrong thing. It is the agent doing the right thing, correctly, over and over, past the point anyone meant it to.

The obvious answer is to test the agent carefully before it ships and trust it once the scope looks right. We think that is the wrong answer, because a careful test checks behaviour against cases somebody thought of. It cannot check against every queue that goes quiet at 2 a.m., every system that is down longer than expected, every loop that looks, from inside the agent, exactly like doing its job.

So every agent we stand up gets both, agreed before it runs its first job, not added after the first invoice makes the case for itself.

How it works, roughly

Alongside the written scope, each named agent carries a budget — tokens, calls, spend — set by the person who owns the process, not by the agent. What it reads, on every run:

• the job it is partway through
• its own spend so far against the limit set for it
• nothing about any other agent's ledger — a budget is scoped the same way a job is

What it gives back, at the limit, is a stop — not a quieter failure buried three log lines down, and not a graceful continuation on borrowed capacity. The agent halts and raises the limit to a person as a decision that needs making, not a number it hopes nobody checks. Raising the limit, or lowering it, is itself a logged choice, made by a person, not an automatic retry the agent grants itself.

AI Transformation · drag to turn