Guide
How LLM API Pricing Works: Tokens, Asymmetry, Waste
LLM APIs bill per token, and output costs 4–5× input. The Asymmetry Audit tells you which half of your bill to attack first.
The short answer
LLM APIs bill per token, and output tokens cost roughly 4–5× input tokens. Every call bills twice, so waste hides in resent prompts and verbose answers. The Asymmetry Audit, output's share of total spend, shows whether to constrain generation or compress input first.
LLM providers like OpenAI, Anthropic, and Google bill through a pay-per-token API model: cost is calculated from the amount of text processed, not from subscriptions or server uptime. Every API call splits into two billing categories — input tokens (everything you send: prompts, chat history, system instructions) and output tokens (everything the model generates). Output tokens are priced roughly 4–5× higher than input tokens across providers, because sequential generation costs far more compute than parallel prompt reading. A token corresponds to roughly 3.5–4 English characters, so 1,000 tokens is about 750 words. Because billing counts every token, waste accumulates invisibly: bloated system prompts resent on every turn, unconstrained verbose outputs, uncached repeated inputs, and frontier models doing work a small model handles. The Asymmetry Audit below — one division — tells you which half of the bill to fix first.
Introduction
Most LLM providers bill through a pay-per-token API economy: instead of flat subscription rates or server uptime, cost is a direct function of text processed and generated. That single design decision shapes everything about production AI economics — and most teams discover its implications on an invoice rather than in a design review. This piece covers the mechanics: what a token is, how the two halves of every API call are priced, where waste hides, and a one-line calculation that tells you which half of your bill to attack.
Why It Matters
Per-token billing means cost scales with behavior, not seats. Two products with identical user counts can have bills an order of magnitude apart depending on prompt design, history handling, and model selection. Understanding the pricing anatomy is the prerequisite for every optimization decision downstream — routing, caching, self-hosting break-evens all start from knowing what a token costs and which tokens dominate your spend.
Core Concepts
Token. The smallest unit of text a language model processes — a word, subword, or character sequence. In English, one token corresponds to roughly 3.5–4 characters, per Anthropic's glossary. Practical conversion: 1,000 tokens ≈ 750 words. (A common inversion — "1,000 words ≈ 750 tokens" — gets the ratio backwards and underestimates costs by ~44%.)
Context window. The total tokens a model can process in one call: your input plus its output. Everything inside it is billed.
The Anatomy of One API Call
Every call bills twice, and the two halves are not priced alike:
| Aspect | Input tokens | Output tokens |
|---|---|---|
| What's counted | Prompt, user messages, conversation history, system instructions, tool definitions | Everything the model generates back |
| Also called | Prompt tokens | Completion tokens |
| Relative price | Baseline | Typically 4–5× input |
| Why | Prompt is processed in parallel | Generation is sequential — each token requires a full forward pass |
Two details in the diagram carry most of the cost consequences. First, the input side is a stack, and only its bottom layer is new: on every conversational turn, the system prompt, tool definitions, and the entire accumulated history are resent and re-billed — the user's new message is typically the smallest slice of what you pay to send. Second, the output block's price premium is physical, not commercial: reading a prompt is a parallel operation over all input tokens at once, while generation produces one token per forward pass, each pass conditioned on everything before it. Providers price the compute difference at roughly 4–5×.
The asymmetry is consistent across providers: On the list prices this site checked against vendor pages on 2026-10-24 (see AI model list prices), Claude Opus 5.5 charges $4 per million input tokens and $20 per million output tokens, and GPT-6 Astra charges $10 and $50: output costs five times input in both cases. Anthropic and OpenAI publish their current rates at anthropic.com/pricing and platform.openai.com/docs/pricing. Output is the premium product.
Where Hidden Waste Accumulates
Because billing tracks every token, inefficient designs compound silently — and the compounding is geometric in conversations, because the input stack from Section 1 is resent whole on every turn:
The chart is illustrative in scale but exact in shape: the dark base is the system prompt, identical and re-billed every turn; the light block is the history, which contains every previous turn's input and output. By turn ten, the tokens the user actually typed that turn are a rounding error on the bar. Four waste patterns drive this and its siblings:
Bloated system prompts. Long background instructions and context documents resent with every user turn. A 3,000-token system prompt in a 20-turn conversation bills 60,000 input tokens before the user has said anything new — the dark base of the chart, twenty times over. The fix hierarchy: cut what isn't needed, move static content into a cacheable prefix, and summarize what remains.
Verbose outputs. Unconstrained generation at 4–5× input pricing. Every filler sentence the model produces is billed at the premium rate — and then billed again as input on every subsequent turn, because outputs join the history stack. Uncapped max_tokens and chatty response formats are direct margin leaks that compound.
No caching. Paying full price to reprocess identical or near-identical inputs. Provider-side prompt caching discounts repeated prefixes — which is precisely the dark base of the chart — while semantic caches skip the call entirely for repeated questions.
Over-powered models. Frontier models (Claude Opus-class, GPT-4o-class) doing categorization or formatting a small model (Haiku-class, mini-class) handles at 10–20× lower per-token rates. The waste multiplies with the other three: a bloated prompt sent to an oversized model on every turn stacks two markups on the same unnecessary tokens.
Mature AI operations attack these through prompt compression, caching layers, and routing simple requests to cheaper models.
The Asymmetry Audit
Everyone lists the waste categories. Nobody tells you which to fix first. The 4–5× price asymmetry does — with one division.
Output share of spend:
- S =
- (T_out × P_out) ÷ (T_in × P_in + T_out × P_out)
- T_in, T_out
- Your monthly input and output token volumes (from your usage logs)
- P_in, P_out
- Your model's published per-token rates
The decision rule:
- S > 0.5 (output-dominated): Constrain generation first. Cap
max_tokens, demand structured or terse output formats, cut conversational filler from response templates. Prompt work is second-order here. - S < 0.5 (input-dominated): Attack input first. Prompt caching, system-prompt compression, history summarization, surgical RAG. Typical of chatbots dragging long histories and retrieval-heavy apps.
- Recompute after each fix — the dashed loop in the flow. Cutting one side flips which side dominates, and the priority flips with it.
An illustrative pass through the arithmetic (shapes, not quotes of any provider's rates): a chatbot logs 40M input tokens and 4M output tokens in a month, at rates of $1 and $5 per million respectively. S = (4 × 5) ÷ (40 × 1 + 4 × 5) = 20 ÷ 60 ≈ 0.33 — input-dominated despite output's 5× price, because the volume skew is steeper than the price skew. The right first move is caching and history summarization, and the intuition many teams get wrong is exposed: the expensive token type is not automatically the dominant cost. A batch-summarization pipeline with the reverse volume shape would land at S ≈ 0.7 and take the opposite branch.
The audit takes five minutes with request-level token logging in place, and it prevents the most common optimization mistake: weeks spent compressing prompts on a bill that generation verbosity is actually driving.
Limitations
- Rates move. Per-token prices and the input/output ratio are provider decisions; every number here should be re-checked against the pricing pages at decision time.
- The token-to-word ratio is language-dependent. Non-English text, code, and dense notation tokenize at different rates; the 3.5–4 character heuristic is English prose only.
- Cached and batch pricing complicate S. Prompt-cache hits and batch-API discounts bill at reduced input rates; compute S from actual spend, not list prices, once discounts apply.
- Worked figures are illustrative. The chart scales and the S example use round numbers to show shape, not quoted rates.
- This covers API billing only. Self-hosted inference has a different cost structure entirely — fixed infrastructure rather than per-token rates.
Final Thoughts
Per-token billing rewards teams that treat cost as an architecture property, not an accounting line. The mechanics are simple — two token types, one steep price asymmetry — but the asymmetry is the operative fact: it means the shape of your traffic, not just its volume, sets your bill. The three visuals in this piece are the same lesson from three angles: the call anatomy shows what's billed, the turn chart shows how it compounds, and the audit flow turns both into an ordered plan. Run the Asymmetry Audit before any optimization sprint.
Evidence in this article Editorial interpretation Our reading of what the evidence means. Judgment, clearly separated from the evidence itself.
Frequently asked questions
How are LLM API costs calculated?
Per token processed: input tokens (everything sent to the model) plus output tokens (everything generated), each billed at its own per-million-token rate.
Why are output tokens more expensive than input tokens?
Generation is sequential — each new token requires a full forward pass through the model — while input prompts are processed in parallel. Providers price the compute difference at roughly 4–5×.
How many tokens is 1,000 words?
Roughly 1,330 tokens in English prose, using the ~3.5–4 characters-per-token heuristic. Equivalently, 1,000 tokens ≈ 750 words.
What counts as input tokens in a chatbot?
Everything resent per turn: the system prompt, tool definitions, and the accumulated conversation history — which is why long chats get progressively more expensive per message.
Does the model's previous answer get billed again?
Yes — every generated response joins the conversation history and is re-billed as input on all subsequent turns, which is why verbose outputs cost twice: once at the output rate, then repeatedly at the input rate.
Sources
- Anthropic, Glossary Anthropic.
- Anthropic, API pricing Anthropic.
- OpenAI, API pricing OpenAI.