AI API costs are like a leaky bucket. Water drains through multiple holes simultaneously, and plugging the tiny ones first is a waste of effort. The difference between engineers who cut their bills in half—or by 90%—and those who fail isn’t the number of tricks they know, but their priority order. Always plug the biggest leaks first. Below is the recommended sequence. Note: specific figures use Claude API pricing as a reference (models and pricing change over time, so check official pricing pages before applying).
Leak 1: Model Overkill — The Largest and Most Common Leak
Most budget waste starts right here: using expensive models for simple tasks. Calling flagship models for tasks like classification, extraction, summarization, or basic formatting is like hiring an excavator to drive a nail.
Based on Claude pricing per million input/output tokens, Haiku 4.5 costs roughly $1/$5, Sonnet 4.6 costs $3/$15, and Opus 4.8 costs $5/$25. Running the same task on a lower-tier model instead of a flagship model drops the unit cost by several multiples. Industry analyses show that simply routing tasks between Haiku and Sonnet based on difficulty can slash total billings by 60–80%.
How to fix: Tier your tasks by difficulty. Send easy, high-volume workloads to the cheapest model, reserving top-tier models exclusively for complex reasoning. Save flagship models only for moments when “a better initial answer is genuinely worth the premium.”
Leak 2: Repeated Context — Plug It with Caching
If you re-send the same system prompt, document, or few-shot examples with every API call, you are paying full price for those input tokens repeatedly. This leak is especially severe in workloads that repeatedly send large, static context, such as RAG systems or coding assistants.
Prompt caching solves this problem. By caching stable prefix content (system prompts, knowledge bases, long documents) on the server, the read cost for reused tokens drops by up to 90% compared to base input tokens. Although cache writing costs about 1.25x (for short-term retention) on the first call, you break even after just one or two reuses. In apps with heavy context repetition, caching alone can reduce costs by 88–95%.
How to fix: Place static content (instructions, documents, examples) at the beginning of your prompt and enable caching. Append request-specific dynamic content at the end. Since caches are sensitive to order, structuring your prompt as ‘Static → Dynamic’ is essential.
Leak 3: Bloated Inputs — Put Context on a Diet
The third leak comes from the habit of “dumping everything in.” Sending entire files, complete chat histories, or full documents with every call explodes your input token count. Crucially, when request inputs cross 200k tokens, long-context premium pricing kicks in, increasing costs even further.
How to fix: Pass only what the model strictly needs. Use retrieval (RAG) to inject relevant excerpts instead of whole documents, compress long chat histories by summarizing older turns, or truncate them using a sliding window. Trimming redundant, wordy sentences from system instructions also yields non-trivial cumulative savings. The cheapest token is the one you never send.
Leak 4: Real-Time Obsession — Cut Costs by Half with Batching
Fourth: using synchronous calls out of habit when real-time processing isn’t required. This applies to tasks that do not demand immediate responses, such as overnight report generation, bulk classification, content cleaning, or scheduled data extraction.
The Batch API plugs this leak. By bundling requests asynchronously, you get results within 24 hours (usually much faster) at a flat 50% discount on both input and output tokens. Because the same model runs the exact same inference, output quality is identical—it is simply scheduled during off-peak hours. Furthermore, batching stacks with prompt caching, allowing combined savings of up to 95% on input tokens for repetitive context workloads.
How to fix: Divide workloads into ‘must be real-time’ and ‘asynchronous.’ Route everything in the latter group to a batch queue. This action alone immediately cuts billing for those jobs in half.
Leak 5: Long Outputs — Outputs Cost 5x More Than Inputs
The final leak is frequently overlooked: across all Claude models, output tokens cost 5 times more than input tokens. For tasks generating long outputs—like code, articles, or reports—the bulk of your bill comes from output tokens. Allowing the model to generate verbose responses wastes your most expensive tokens.
How to fix: Set realistic output limits (max_tokens) and constrain response formatting using instructions like “be concise,” “focus only on key points,” or “format as a table.” Use structured outputs (such as JSON) and stop sequences to eliminate fluff. If using reasoning models, adjust thinking token budgets and effort levels according to task difficulty to prevent wasting excessive thinking budget on simple tasks. For output-heavy workloads, combining model right-sizing (Leak 1) with output controls (Leak 5) delivers the biggest impact.
Before Plugging Leaks: Measure Where It’s Leaking First
Once you follow this sequence, one golden rule remains: you cannot optimize what you do not measure. Log input, output, and cache tokens per request to identify the top operations consuming your budget. Some workloads are input-heavy (where caching and context diets are the answer), while others are output-heavy (where model right-sizing and output controls matter most). Pinpointing the primary driver tells you which of the five leaks requires the most focus.
To summarize: plug the leaks by size—right-size your models, cache repeated context, trim bloated inputs, offload non-real-time jobs to batches, and control output length. Following these five steps in order lets you achieve the exact same results at a completely different order of magnitude in cost.
⚠️ The pricing, discount rates, and model names in this post are accurate as of August 2026. LLM pricing structures change frequently. Always check the official pricing pages of your API provider for the latest figures before implementation.
Frequently Asked Questions
Which leak should I plug first?
Model overkill. Using flagship models for tasks like classification, extraction, summarization, or simple formatting is the most common and largest source of waste.
When is prompt caching most effective?
When you re-send the same system prompt or documents with every API request. It is especially impactful for workloads that repeatedly transmit large, static context, such as RAG systems or coding assistants.
Why do output tokens matter so much?
Across all Claude models, output tokens cost 5 times as much as input tokens. Tasks generating long outputs, such as code, articles, or reports, skew your bill heavily toward output costs.
When should I use the Batch API?
For tasks that do not require an immediate response, such as overnight report generation, bulk classification, content cleaning, or scheduled data extraction.
Leave a Reply