Retry budgets: why one global limit wasn’t enough

A single retry setting looked simple until one flaky upstream started eating whole runs. This is how we moved to per-tool retry budgets, and why the default got stricter.

The problem with one number

For the first year, every agent had one retry limit: three attempts per step. It was easy to explain and it hid a nasty failure mode. When a single slow dependency degraded, every step that touched it retried three times, and a run that should take four seconds took forty.

What it looked like in traces

• Runs that finished, but at ten times the usual cost.

• Retries stacking on a tool that was never going to recover in time.

• Healthy tools starved of budget because the run hit its overall timeout.

Budgets per tool

Now each tool declares its own budget: how many retries, how long to back off, and whether a failure should stop the run or let the agent route around it. Search can retry freely. Payments never retry automatically.

The right retry policy depends on what the tool does when it is called twice, not on how often it fails.

Idempotency first

A tool that isn’t safe to call twice should not retry, full stop. We now ask for an idempotency flag when you register a tool, and the runtime refuses to retry tools without one.

Results

After the change, p95 run time dropped 38% for teams with one or more flaky dependencies, and cost per run fell 22%. Failures went up slightly, which is the point: a fast, honest failure beats a slow, expensive success that nobody trusts.