Circuit Breakers for LLM Agents: Stopping Runaway Tool Loops Before They Burn Budget or Cause Side Effects

A familiar failure pattern shows up the moment agent prototypes meet real traffic: the model does not fail loudly, it fails expensively. A user asks a seemingly simple question, the agent starts reasoning, calls one tool, then another, retries the first tool because the output format looked odd, reformulates the same query three different ways, opens a second retrieval path, decides to verify its own answer, then gets stuck in a loop between “search,” “fetch,” and “summarize.” Nothing is technically broken. Every component is doing what it was asked to do. But the session takes 45 seconds, burns through a meaningful amount of tokens, hits multiple paid APIs, and maybe worse, triggers side effects twice because the model could not reliably distinguish “already done” from “still pending.”

That is the LLM-agent version of a cascading systems failure. Not a single dramatic outage, but an accumulation of small autonomous decisions that become cost, latency, correctness, and safety problems.

In conventional distributed systems, we learned long ago to build circuit breakers around flaky dependencies and dangerous operations. We cap retries. We fail fast. We separate reads from writes. We isolate blast radius. LLM agents need the same discipline, but with a twist: the source of unreliability is not just downstream services. It is also the planner itself. The model can generate an unbounded sequence of plausible next actions, and those actions can be expensive, slow, or harmful even when each individual tool call is valid.

If you are shipping agentic systems into production, circuit breakers are not a nice-to-have safety layer. They are the mechanism that converts “open-ended autonomy” into “bounded execution.” Without them, your agent architecture effectively has no governor.

This article lays out a production-oriented approach to circuit breakers for LLM agents: what to trip on, where to enforce limits, how to contain side effects, what to do after a breaker trips, how to instrument the whole system, and which evals catch runaway plans before users or downstream systems do.

The recurring pattern: agents fail by drifting past invisible thresholds

The reason runaway loops are so common is that teams often define success criteria at the task level but forget to define failure boundaries at the execution level.

A typical prototype is built around a simple loop:

Send conversation and tool results to the model.
If the model asks for a tool, execute it.
Append result.
Repeat until the model returns a final answer.

This looks harmless in demos because the tasks are short, tools are clean, and the operator is watching. In production, that same loop collides with messier realities:

Retrieval tools return ambiguous or noisy evidence, prompting repeated searches.
External APIs fail transiently, inviting retries.
The model hallucinates missing parameters and repeatedly reformulates calls.
The agent lacks a clear terminal condition and keeps “improving” an already good answer.
Write tools succeed, but the confirmation signal is weak, causing duplicate actions.
The cost of one more step feels locally justified to the model, while globally violating business budgets.

This is what makes agent failure structurally different from ordinary request failure. The danger is not only “the tool errored.” The danger is “the planner keeps consuming resources while appearing rational at each step.”

In practice, runaway behavior usually clusters into a few categories:

First, search and retrieval spirals. The agent keeps broadening or narrowing search because retrieved evidence is incomplete. This is common in RAG-heavy systems where the planner treats uncertainty as a cue to search again.

Second, retry storms. The model alternates between the same few tools, often after parse errors, partial failures, or timeouts. Sometimes the downstream system is degraded. Sometimes the tool schema is under-specified. Either way, the planner amplifies the issue.

Third, self-verification loops. The agent answers, then decides to verify, then decides the verification is insufficient, then adds more verification. This is particularly common when prompts encourage carefulness without bounding it.

Fourth, duplicate side effects. The agent sends the same email twice, creates two tickets, issues two refunds, or posts repeated messages because “write succeeded” and “write acknowledged” were not cleanly modeled.

Fifth, multi-agent amplification. A coordinator delegates to sub-agents, which each invoke their own loops. Costs and latency multiply nonlinearly, while responsibility for termination becomes unclear.

Once you recognize these as recurring execution patterns rather than isolated bugs, the engineering response becomes clearer: add explicit circuit breakers at the runtime layer, not just nicer prompts.

Why the naive approach fails

The naive mitigation strategy is prompt-only governance. Teams tell the model things like:

“Use tools only when necessary.”
“Do not make excessive API calls.”
“Avoid repeated searches.”
“Be concise and efficient.”

These instructions help at the margin, but they are not enforcement. When the model faces ambiguity, the prompt competes against other objectives: answer accurately, be thorough, recover from errors, satisfy the user, follow the demonstrated tool-use pattern. The result is predictable: the model often chooses another step.

Another common naive strategy is a single max-iteration cap, for example, “stop after 10 tool calls.” Better than nothing, but too coarse for production. Ten cheap retrieval calls are very different from ten expensive web-agent browser actions or ten write attempts into a ticketing system. A flat step cap also fails to account for session context: you may want a premium internal analyst workflow to take more steps than a public customer support session.

A third weak approach is tool-specific retry logic implemented in isolation. For example, a search tool retries twice, a CRM tool retries once, and a payments tool has its own timeout. This helps service resilience, but it does not solve planner-level runaway behavior. The model can still route around those limits by switching tools or repeatedly invoking the same tool with slightly modified arguments.

The final naive strategy is post hoc alerting: monitor costs and incident reports after the fact. Useful for learning, useless for prevention. By the time you discover that an agent spent $18 solving a $0.20 problem or created 50 duplicate tickets, the breaker should have tripped minutes earlier.

A better approach: bounded autonomy through layered circuit breakers

The right mental model is not “make the model smarter so it stops itself.” It is “treat the model as an untrusted but useful planner operating inside a constrained execution sandbox.”

That means circuit breakers should exist at multiple layers:

Per-step limits: constrain what can happen in one model turn or tool invocation.
Per-tool limits: bound retries, spend, latency, and side effects for each tool category.
Per-session limits: cap total steps, wall-clock time, token usage, and external spend.
Per-user or per-tenant limits: prevent abuse and contain account-level blast radius.
Global kill switches: stop entire workflows or tool classes during incidents.

The key design principle is simple: the model may propose the next action, but the runtime decides whether that action is still allowed.

A practical reference architecture

In production, the cleanest pattern is to separate planning from execution control. Do not let the agent loop live entirely inside application code with ad hoc if-statements. Give it a policy-enforcing runtime.

A reference architecture looks like this:

Agent frontend: receives the user request and selects the workflow or agent profile.
Planner model: generates tool calls or final answers.
Execution controller: validates proposed actions against policy and current budget state.
Tool gateway: standardizes tool invocation, auth, idempotency, timeouts, and logging.
State store: tracks per-session counters, costs, timestamps, side-effect attempts, and breaker state.
Policy engine: evaluates trip conditions and decides allow, deny, degrade, or terminate.
Observability pipeline: emits traces, metrics, and structured events.
Operator console: exposes overrides, approvals, and incident controls.

The critical component is the execution controller plus policy engine. Every model-emitted action should flow through it.

Conceptually:

Model proposes a tool call.
Runtime estimates the incremental cost, risk, and latency exposure.
Policy engine checks current session/tool/user/global budgets.
If allowed, tool gateway executes with strict timeout/idempotency rules.
Result updates state store.
If thresholds are crossed, breaker trips and the runtime selects a fallback.

This extra indirection is what turns agents from “clever loops” into governable systems.

What should trip a circuit breaker

The most important design question is what conditions should cause the runtime to stop or degrade execution. In practice, you want a mix of hard limits and pattern-based limits.

Cost thresholds

The most obvious breaker is spend. But define it carefully:

Token spend: prompt + completion + reasoning tokens where applicable.
External tool spend: search APIs, scraping APIs, browser sessions, geocoding, OCR, etc.
Internal compute spend: GPU-backed embedding or custom inference if significant.

Use multiple scopes:

Per-step estimated cost ceiling.
Per-tool budget within a session.
Total session budget.
Daily per-user/per-tenant caps.

For example, a support agent may have a $0.08 median target, $0.25 soft session cap, and $1.00 hard cap. A research workflow might have a higher ceiling but still separate retrieval budget from synthesis budget.

A useful pattern is soft versus hard budgets. Soft budget crossing can trigger degradation, such as downgrading to a cheaper model or limiting further retrieval. Hard budget crossing terminates tool execution entirely.

Latency thresholds

Users care about elapsed time more than internal reasoning quality. Long agent loops destroy trust.

Track:

Per-tool timeout.
Cumulative wall-clock session limit.
Time since last meaningful progress.
Queueing delay in external dependencies.

That last one matters. Sometimes the agent is not looping because it wants to; it is blocked on slow tools. Either way, from the user perspective, the workflow is runaway.

I recommend a “progress timeout” in addition to a wall-clock timeout. If the system has executed N steps without producing a materially new state transition—new evidence retrieved, action completed, or answer confidence improved—trip or degrade.

Step and recursion thresholds

The simplest useful breaker remains total tool-call count, but it should be more expressive than a flat max.

Track:

Total steps.
Consecutive calls to the same tool.
Repeated calls with semantically similar arguments.
Re-entry into the same plan node or sub-agent.
Fan-out count for multi-agent delegation.

Repeated-tool detection is especially valuable. Many runaway loops are not “10 different actions,” but “the same action 6 times with tiny argument changes.” Use normalized arguments or embedding similarity to detect near-duplicates.

Risk thresholds

Not all tools are equal. A read from a knowledge base and a refund issuance should not share the same breaker profile.

Assign tool risk classes, for example:

R0: pure read-only, cached or internal retrieval.
R1: external reads, search, browsing, scraping.
R2: internal mutations with reversible effects, like draft creation.
R3: user-visible writes, ticket creation, message sending.
R4: financial, security, compliance, or destructive actions.

Then define stricter conditions for higher-risk classes:

Fewer allowed retries.
Stronger idempotency requirements.
Approval gates or two-phase commit for R3/R4.
Lower tolerance for ambiguous arguments.
Mandatory fallback to human review when confidence is low.

Breaker design should be risk-aware, not merely iteration-aware.

Pattern-based trip conditions

The best production systems do not only count; they recognize suspicious trajectories.

Useful patterns include:

Same tool called repeatedly with low-entropy argument changes.
Alternation loop: search -> fetch -> search -> fetch.
Tool errors of the same class repeated beyond threshold.
Contradictory plan revisions, e.g., repeated “need more data” after adequate evidence.
Duplicate side-effect attempts against the same entity.
Hallucinated parameter repair loops.

These are often more predictive of runaway behavior than raw step count.

Per-tool and per-session budget design

The cleanest way to reason about budgets is to define them as an explicit contract per workflow.

For each agent workflow, specify:

Allowed tools.
Max tool calls by tool.
Max session steps.
Max wall-clock duration.
Soft and hard token budget.
Soft and hard external API spend.
Max write attempts.
Approval requirements for certain risk classes.
Fallback policy when any threshold trips.

Here is a representative support-agent budget:

Model: mid-tier fast model for planning and response.
Max session wall clock: 12 seconds.
Max steps: 6.
Knowledge base search calls: 2.
CRM read calls: 2.
Ticket write calls: 1.
Total token budget soft/hard: 12k / 24k.
External spend soft/hard: $0.05 / $0.15.
On breaker trip: return best-effort answer, attach “needs human follow-up” if unresolved.

Compare that to a sales-ops automation agent:

Max wall clock: 45 seconds.
Max steps: 12.
CRM write calls: 2.
Email draft write calls: 1.
Email send calls: 0 without approval; 1 with approval token.
Duplicate write protection window: 15 minutes by entity and action hash.
On breaker trip: persist draft, notify operator, do not send externally.

The point is that budgets should reflect business value, user expectations, and side-effect risk, not generic agent enthusiasm.

Side-effect containment: where most real damage happens

If I had to prioritize one area, it would be containment of side effects. Excess retrieval spend is annoying. Duplicate writes to production systems create incidents.

There are four controls that matter most.

First, idempotency keys everywhere. Any tool that creates or mutates something should accept an idempotency key derived from session ID, action type, target entity, and semantic arguments. If the agent retries, the downstream system should return the previous result rather than execute again.

Second, two-phase write patterns. Separate “prepare” from “commit” for higher-risk tools. The model can draft an email, stage a refund request, or assemble a ticket mutation plan, but the actual commit either requires explicit approval, stronger confidence conditions, or a secondary policy check.

Third, write-once semantics within a session for sensitive actions. For example, allow at most one refund attempt, one password reset initiation, or one outbound message send unless an operator override is present.

Fourth, compensating actions and auditability. If a write slips through and a later breaker trips, you need enough structured logging to detect whether rollback is possible and what user-visible follow-up is needed.

A lot of agent teams make the mistake of giving write tools the same interface shape as read tools. That is convenient for the planner and dangerous for the system. Writes need a stricter contract: stronger validation, deterministic execution semantics, and richer acknowledgments than “success: true.”

Fallback behaviors after a breaker trips

Stopping execution is only half the design. You also need a controlled user and operator experience after the stop.

Good fallback patterns include:

Best-effort answer with explicit limitation

If the agent has enough evidence to provide a useful partial answer, do so. Say what was completed and what was not. This works well for research, support, and internal knowledge workflows.

Graceful degradation to a simpler path

If the richer agent workflow exceeds budget, fall back to a narrower non-agentic path: single-pass RAG, FAQ lookup, rules-based response, or a cheaper smaller model with no tools.

Escalation to human review

For operational workflows, breaker trips should often hand the case to an operator with session trace, attempted actions, and structured reason for trip.

Staging rather than committing

When writes are involved, a breaker trip should preserve work product in a reversible form: draft email, pending ticket, staged workflow plan, approval request.

User-visible retry later

If the issue is caused by degraded dependencies, tell the user the action could not be completed now and preserve enough state to retry safely later.

The wrong fallback is to let the model improvise after the breaker trips. Once limits are exceeded, autonomy should narrow, not broaden.

Operator overrides and kill switches

No matter how good your automatic controls are, production needs human override paths.

At minimum, support:

Per-session override to increase budgets temporarily.
Per-tool disable switch.
Workflow-wide kill switch.
Tenant-specific quarantine.
Approval injection for staged high-risk actions.

Overrides should be explicit, audited, time-bounded, and ideally require a reason code. The failure mode to avoid is “operators silently relax all limits because the breaker was noisy.” If overrides become routine, your thresholds or workflow design need fixing.

Observability: the difference between breaker policy and superstition

Many teams can tell you total token cost per day. Far fewer can explain why agents crossed thresholds, which tools contributed, or what pattern preceded the trip. Without structured observability, breaker tuning becomes folklore.

Instrument each session as a trace with spans for:

Model calls.
Tool proposals from the planner.
Policy decisions.
Tool executions.
Side-effect staging and commit.
Breaker state transitions.

Emit structured attributes such as:

Session ID, workflow ID, user/tenant.
Model name and version.
Prompt template version.
Tool name, risk class, and arguments hash.
Estimated and actual token cost.
External API cost.
Elapsed and queueing latency.
Retry count.
Duplicate-call similarity score.
Breaker rule triggered.
Fallback path selected.

Key metrics to monitor:

Breaker trip rate by workflow.
Mean and p95 steps per session.
Mean and p95 tool calls by tool.
Duplicate write prevention count.
Cost per successful task.
Latency to meaningful progress.
Sessions terminated for repeated-call patterns.
Human escalation rate after breaker trip.

One practical recommendation: keep both pre-execution and post-execution events. You want to know not only what was done, but what the model attempted to do and why it was blocked.

Implementation details: policy enforcement in code

A robust implementation usually has three concepts:

Budget ledger

A mutable per-session object that accumulates spend, steps, time, and risk events.

Policy rules

Pure functions that inspect proposed action plus current ledger and return allow, deny, degrade, or require approval.

Execution outcome reducer

A function that updates the ledger after each model call or tool result and emits state transitions.

Pseudocode for the core loop might look like this:

python
while True:
    if policy.hard_stop(session_ledger):
        return fallback(session_state, reason=policy.stop_reason)

    model_output = planner(session_state)
    ledger.add_model_usage(model_output.usage)

    if model_output.final_answer:
        return respond(model_output.final_answer)

    action = model_output.tool_call
    decision = policy.evaluate(action, session_ledger, session_state)

    if decision == "deny":
        session_state.add_system_message(render_denial(action))
        if policy.should_terminate_on_denial(action):
            return fallback(session_state, reason="policy_denial")
        continue

    if decision == "degrade":
        action = degrade_action(action)

    if decision == "require_approval":
        return stage_for_approval(session_state, action)

    result = tool_gateway.execute(
        action,
        timeout=policy.timeout_for(action.tool),
        idempotency_key=make_idempotency_key(session_state, action),
    )

    session_ledger = reduce_outcome(session_ledger, action, result)
    session_state.add_tool_result(action, result)

There are a few implementation details worth emphasizing.

First, estimate before executing. For some tools you can approximate incremental cost and latency before making the call. If a browser action is likely to exceed remaining budget, deny it up front.

Second, normalize tool arguments. Store canonicalized forms to detect duplicates. For example, strip whitespace, sort filter params, lowercase known fields, and hash semantically equivalent payloads.

Third, define “meaningful progress.” It can be as simple as a state transition taxonomy: retrieved new document IDs, fetched new entity fields, staged write action, received durable acknowledgment, or improved answer confidence. If no progress has occurred in several steps, that is itself a breaker signal.

Fourth, keep the model informed, but not in control. The runtime may inject a system or tool result message saying, for example, “Search budget exhausted; continue without additional retrieval.” This helps the model conclude gracefully. But the runtime still enforces the rule.

Model and tool choices: cost and latency tradeoffs

Circuit breakers are necessary partly because model and tool choices create different failure surfaces.

Large frontier models are often better planners, but they can also justify longer chains of action and carry higher token cost when loops occur. Smaller fast models may loop less expensively, but they can fail to recognize completion or hallucinate tool arguments more often.

In practice, useful patterns are:

Use a fast, cheaper model as the default planner for bounded workflows where tools are well structured.

Reserve more capable expensive models for:

high-ambiguity tasks,
long-context synthesis,
sessions that have already hit difficulty signals,
human-in-the-loop draft generation rather than autonomous execution.

For tools, prefer APIs with deterministic outputs and explicit status semantics over “scrape and infer” when possible. The cleaner the tool contract, the fewer repair loops the planner enters.

Also consider the hidden cost of verification. Teams often add extra model calls to “judge” whether an action succeeded or whether enough evidence has been collected. This can improve reliability, but it can also create a meta-loop where the system spends significant budget evaluating its own progress. If you add verifier models, they need budgets too.

A practical comparison framework:

Planner quality: how often does the model choose the shortest valid path?
Tool argument reliability: schema adherence and parameter correctness.
Stop reliability: how often does the model terminate appropriately without extra prompting?
Recovery behavior: does it retry sensibly after a tool error?
Cost under adversity: not just average cost, but cost in worst 5% sessions.

That last measure is where many “best” models disappoint in production. Circuit breakers should be tuned to tail behavior, not just happy-path medians.

Evaluation strategy: test for runaway behavior explicitly

Most agent eval suites over-focus on task success and under-focus on bounded execution. You need evals that ask not just “did it solve the task?” but “did it solve the task within acceptable operational limits?”

I recommend four layers of evaluation.

Synthetic adversarial workflow tests

Create scenarios designed to induce loops:

Retrieval returns partial but plausible evidence.
Tool responses vary slightly across retries.
APIs time out once, then recover.
Write acknowledgments are delayed or ambiguous.
Conflicting search results appear.
A sub-agent returns low-confidence outputs repeatedly.

Measure whether breakers trip at the intended points and whether fallback is correct.

Historical trace replay

Take real sessions from logs, especially costly or incident-adjacent ones, and replay them through newer policies and models. This is one of the highest-leverage evaluation methods because it captures realistic user prompts and tool messiness.

Property-based policy tests

Treat breaker logic as a policy engine deserving unit tests. For example:

A write tool cannot execute without idempotency key.
No more than one R4 action per session absent override.
Total projected spend cannot exceed hard cap.
Three near-duplicate search calls should trigger degradation.

This sounds mundane, but it is how you prevent silent regression in safety-critical control logic.

Tail-cost and tail-latency benchmarking

Do not evaluate only average success and average cost. Track:

p95/p99 steps,
p95/p99 token usage,
p95/p99 external spend,
p95/p99 wall-clock time,
rate of duplicate-write prevention,
rate of fallback correctness.

Your breakers exist for the tail. Evaluate the tail directly.

A concrete eval scorecard might include:

Task success rate.
Bounded success rate: success within cost, latency, and step limits.
Safe termination rate: percent of failed sessions that terminate without disallowed side effects.
Duplicate side-effect rate.
Human-escalation appropriateness.
False positive breaker rate: sessions stopped too early.
False negative breaker rate: runaway sessions not stopped in time.

Notice the tradeoff: an aggressive breaker can reduce cost while harming success. The goal is not “trip often.” The goal is “trip when continued autonomy has negative expected value.”

How to tune thresholds without crippling the agent

Threshold tuning is where theory meets uncomfortable business tradeoffs.

Start with observational baselines from real or staging traffic:

Distribution of steps by workflow.
Distribution of token and tool spend.
Common repeated-action patterns.
Where successful sessions typically terminate.
Which tool failures are transient versus structural.

Then set thresholds just above healthy behavior and inspect what gets caught. For instance, if 95% of successful sessions finish in 4 steps, a hard cap at 20 is not meaningful. Start with 6 or 8 and see what breaks.

Use progressive enforcement:

Log-only mode for new rules.
Soft enforcement with degradation.
Hard stop once false positive rate is acceptable.

Separate policy by workflow maturity. New experimental workflows should have tighter caps and no high-risk writes. Mature workflows with strong observability and idempotency can earn more autonomy.

One effective pattern is budget shaping across the session:

Early steps are cheap and exploratory.
Later steps require stronger justification.
Write actions consume disproportionate remaining budget.

This mirrors how human operators work: broad search first, then narrowing, then cautious action.

Common mistakes

A few mistakes show up repeatedly.

Relying on the model to self-report confidence or completion. Helpful signal, not a control plane.

Using only one generic breaker for all workflows. A customer support assistant and a finance operations agent should not share identical limits.

Ignoring side-effect semantics. If your write tool cannot guarantee idempotency or clear acknowledgments, your breaker policy is standing on weak ground.

Treating human escalation as failure. For high-risk workflows, escalation is often the correct successful containment path.

Failing open on policy service outages. If the breaker subsystem goes down, the safe default should usually be restricted execution or no writes, not unlimited autonomy.

Not versioning policy with prompts and workflows. Breaker behavior is part of the product and should be tracked as such.

The operational mindset shift

The biggest shift teams need to make is this: an LLM agent is not one component. It is a distributed system whose planner happens to be stochastic.

Once you see it that way, circuit breakers stop feeling like a niche safety feature and start looking like standard production engineering. You would never let a microservice retry an expensive dependency indefinitely, mutate external systems without idempotency, and exceed tenant budgets just because “the code thought it still needed more information.” Do not grant an LLM agent that privilege either.

Good agent systems are not the ones with the most autonomy. They are the ones with the best-shaped autonomy: enough freedom to solve useful problems, enough control to stop before costs, latency, or side effects outrun value.

The practical takeaway

If you are implementing this next week, do not start by debating sophisticated agent cognition. Start with controls.

Ship a budget ledger. Ship a policy engine. Classify tools by risk. Require idempotency for writes. Define soft and hard session budgets. Instrument repeated-call patterns. Implement deterministic fallback behavior. Replay bad traces until the breakers catch them.

Then, and only then, expand autonomy.

That is how you keep agents useful without letting them become expensive little incident generators.