Control AI cost at scale

Most of your AI bill comes from a few predictable places: too many tokens per call, the wrong model doing easy work, oversized contexts you pay to re-read, and retries that quietly run the same expensive request twice. Fix those four and spend drops a lot without touching output quality.

This guide walks each cost source, then gives concrete levers. The theme throughout: cost is a design choice, not a fixed price. You decide how much context to send, which model handles which task, and how often work repeats.

How does AI billing work?

A model bills per tokenA token is a chunk of text, roughly 3 to 4 characters of English. Models read and write in tokens, not words., split into input and output. Input is everything you send: the system prompt, the conversation so far, retrieved documents, tool results fed back in. Output is what the model writes back. For the fundamentals of how this pricing works, see token usage and cost.

Two facts shape everything else:

• Output is more expensive than input. For most Claude models output tokens cost roughly five times input tokens, so a chatty model that writes long answers costs more than the token count alone suggests.
• Input grows on its own. In any loop, each turn re-sends the whole conversation plus every tool result so far. A ten-turn agent run does not pay for one context, it pays for a context that gets bigger every step.

That second point is why a system that looks cheap in a single test can be alarming in production. You are not billed per task, you are billed per token, and tokens accumulate.

Why is my AI bill so high?

Four sources cover almost every surprising invoice.

Tokens you did not need to send

The most common waste is context that is along for the ride. A system prompt with ten examples when two would do. A full file pasted in when the model needed twenty lines. An entire chat history carried forward when the task only depends on the last message.

Every one of those tokens is paid for on every turn that includes it. In a loop, that multiplies.

The wrong model for the job

Running your most capable model on work a smaller one handles fine is pure overspend. Classifying a support ticket into one of three buckets does not need the same model that refactors a service. The capable model might cost five times more per token and produce an identical label.

Oversized contexts you re-read

When an agent keeps appending tool results to one long context, every later turn pays to re-read all the earlier ones. A search that returns 5,000 tokens of results is not a one-time cost. If it sits in context for eight more turns, you pay for it nine times.

Retries that double the work

Retries are necessary, a transient error or a rate limit should not kill a run, but a naive retry re-sends the entire request. If that request was a 100,000-token agent turn, a retry costs another 100,000 tokens. Retries on the most expensive calls are where retry spend concentrates.

A worked example

Picture a support agent for an online store. A customer asks a question. The agent searches an order database, reads a returns policy document, and writes a reply. Here is how a careless version spends, and where each cost hides.

• The system prompt is 3,000 tokens because it carries twelve few-shot examples. Every single call pays for all twelve.
• The returns policy, 8,000 tokens, is pasted into context in full on every request, even when the question is about shipping.
• The agent runs your largest model for every step, including the part where it just classifies the question as "shipping" versus "returns."
• The order search returns 6,000 tokens of raw rows, and they sit in context for the rest of the run.

None of these is dramatic on its own. Together they turn a question that needs maybe 2,000 tokens of real input into a 20,000-token call, run on the most expensive model, repeated for every customer. At ten thousand customers a day, the gap between the careful and the careless version is the whole bill. The rest of this guide is the careful version.

Lever 1: send fewer tokens

The cheapest token is the one you never send. Before any clever optimisation, trim the input.

• Cut the system prompt to what the task needs. Examples earn their cost only if they change the output. Two well-chosen examples usually beat ten.
• Retrieve, do not paste. Instead of dropping a whole document into context, fetch the relevant section. A search step that returns the right 500 tokens is cheaper than 50,000 tokens of "just in case." See RAG fundamentals for the retrieval pattern.
• Summarise long histories. In a long conversation or agent run, replace old turns with a short summary once they stop being relevant. You keep the thread without paying to re-read every word.
• Return distilled results from sub-agents. A worker agent should hand back its conclusion, not its entire transcript. This keeps the orchestrator's context small. See multi-agent architectures.

You can ask Claude Code to do this audit for you on a prompt you already have:

Lever 2: match the model to the task

Not every step needs your strongest model. A practical pattern is to route by difficulty: a small, fast model handles classification, extraction, and routing, and a larger model handles reasoning, planning, and code.

The savings come from volume. If 80 percent of your calls are classification and routing, moving them to a small model can cut overall spend dramatically while the hard 20 percent keep the quality you need. In Claude Code you can set or switch the model per session, so you can keep routine work on a smaller model and reserve the larger one for the hard tasks.

A word of caution: do not downgrade blind. Pick a model per task by testing it on real examples, not by guessing. A cheap model that gets the wrong answer and forces a retry or a human fix ends up costing more. Measure quality first, then push work down to the smallest model that still passes.Common gotchaWatch for silent downgrades too: a cheaper model that returns slightly worse output without erroring. Nothing fails, so the cost saving looks clean while answer quality quietly drops.Track an accuracy metric per task, not only spend, so a quality regression is visible before users feel it.

Lever 3: cache the stable parts

If you send the same large prefix on many calls, a long system prompt, a fixed set of instructions, a reference document, you can avoid paying full price to re-process it every time. Prompt cachingReusing an already-processed prefix of the prompt so the model does not re-read it on every call stores that prefix so later calls read it back cheaply.

The economics are specific and worth knowing. A cache read costs about a tenth of the normal input price. Writing to the cache costs slightly more than a normal read, 1.25 times the base input price for the default 5-minute cache, or 2 times for the 1-hour cache. So caching pays off when the same prefix is reused enough times that the cheap reads outweigh the one slightly pricier write.

Put the stable content at the front and the variable content at the end, so the cacheable prefix stays identical across calls. In Claude Code this happens automatically when you keep your context stable across a session. The full mechanics and the habits that keep it working are in prompt caching.Common gotchaPeople often turn on caching and then slip a timestamp, a per-user greeting, or a "current date" line into the system prompt at the top.That one moving byte changes the prefix every call, so the cache never hits and you pay the write premium for zero reads, ending up more expensive than no caching at all. Pin the variable bits to the end of the prompt and keep the cached prefix byte-for-byte identical.

Lever 4: batch the work that can wait

Not every job needs an answer in seconds. Nightly reports, bulk classification, data enrichment, and evaluation runs can tolerate a few hours of latency. For those, asynchronous batch processing runs the same requests at half the per-token price.

The trade is latency for cost: you submit a large set of requests and collect results later rather than waiting on each one. If a workload is not user-facing and not time-critical, batching it is close to free money. The discount applies to both input and output tokens.

A simple rule: if a human is waiting on the screen, run it live. If a job runs on a schedule or processes a queue, batch it.

Lever 5: make retries cheap, not expensive

Retries should recover from failure without doubling your bill. A few habits keep them cheap:

• Back off and cap. Wait a little longer before each retryTrying a failed operation again after a short, increasing delay, on the assumption the problem was temporary and set a maximum number of attempts. Infinite retries on a failing service burn tokens for no result.
• Retry the smallest failing unit. If one step in a pipeline fails, retry that step, not the whole run. Checkpoint after expensive steps so a failure resumes from the last good state instead of starting over.
• Do not retry non-transient errors. A malformed request or a refusal will fail the same way every time. Retrying it just pays twice for the same failure. Retry timeouts and rate limits; fix everything else.

How do I stop a runaway AI bill?

Every lever above reduces average cost. Budgets stop the worst case. The failure mode for agents is a run that loops far longer than expected, spawns too many sub-agents, or gets stuck searching forever. A budget turns an open-ended invoice into a bounded one, and works hand in hand with the guardrails for a production AI system.

• Per-run token ceiling. Stop a run that exceeds its token budget instead of letting it run away. Surface a partial result and the reason it stopped.
• Iteration cap. Bound how many turns a loop may take and how many sub-agents it may spawn. Most runaway cost is a missing stop condition.
• Alerts on spend. Track daily and per-task spend and alert when it crosses a threshold, so you find a regression in hours, not on the monthly invoice.

You can have Claude Code add these guards to an existing agent loop:

Measure before you optimise

Do not guess where the money goes. Trace it. Log token counts per call, broken down by step, and tag each call with the task it served. Once you can see which step dominates, the right lever is obvious. Often a single step, an oversized retrieval, a too-capable model on a high-volume path, accounts for most of the bill.

A useful starting question to put to Claude Code while you have the trace open:

Mistakes that quietly cost money

A few patterns recur in systems that overspend. Each is easy to miss because nothing breaks, the bill just runs high.

• Caching a prefix that is not stable. If anything near the front of the prompt changes between calls, a timestamp, a shuffled example order, a per-user note, the cache never hits and you pay the write premium for nothing. Keep the variable parts at the end.
• Summarising too aggressively. Cutting history saves tokens, but if you drop a detail the task still needs, the model asks for it again or gets it wrong, and the redo costs more than the tokens you saved. Summarise what is stale, keep what is load-bearing.
• Choosing a model by price alone. The cheapest model that fails the task is not cheap once you add a retry, a fallback to a bigger model, or a human fix. Compare cost per correct answer, not cost per token.
• No per-task tagging in your traces. If your logs show total spend but not which task drove it, you cannot find the expensive path. Tag every call with its task before you need to.
• Retrying the whole run on a single step failure. One flaky tool call should not re-run the eight steps before it. Checkpoint and retry the failing unit.

What order should I apply these levers in?

Work cheapest-first, because the early levers cost nothing and often dominate:

1. Trim context. Free, and usually the biggest win.
2. Right-size models. Move high-volume easy work to a smaller model.
3. Cache stable prefixes. Pays off wherever a large prefix repeats.
4. Batch deferrable work. Half price for anything that can wait.
5. Tame retries and set budgets. Caps the worst case.

None of these trades quality for cost when applied with judgement. Trimming context removes noise, which often improves answers. Right-sizing models keeps the hard work on the strong model. The goal is not the cheapest possible system, it is a system that spends money only where spending it buys something. Measure, apply the lever that fits, and keep the budget in place so a bad run stays a bad run, not a bad month.