LEARN / OUTPUT TOKENS

Why output tokens cost more than input

PUBLISHED 2026-08-04 · BY Llorenç Ballester — Chief Burn Officer

The prompt is visible before you send it, so engineers naturally blame input when the invoice arrives. The reply is easier to underestimate: output is commonly priced above input, may contain reasoning tokens you never see, and grows fast when an agent helpfully rewrites 800 unchanged lines.

Input and output are two different billing lines

A text-generation request has at least two independently priced parts: the tokens you send and the tokens the model generates. Most API models charge more for output. Claude Sonnet 5 is $2/MTok input and $10 output — 5x. GPT-5.6 is $5 in / $30 out — 6x. The multiplier is model-specific and changes, which is why remembered ratios belong in pub conversations, not production forecasts. Current rates should come from the live board, not a spreadsheet copied from a launch post six months ago.

The useful baseline is cost = uncached input × input rate + cached input × cached rate + output × output rate, with separate terms wherever the provider bills cache reads or writes differently. “Total tokens” is fine for context-window analysis but a poor billing metric: it quietly treats differently priced work as equivalent. A short-looking answer can still carry non-visible generated usage, while a long cached prompt can be cheaper than its raw token count suggests.

Generation is where verbosity becomes an invoice

Output grows whenever the model must serialize something: prose, JSON, source code, tool arguments, tables, or three alternatives because the prompt asked for “a few options.” A 100-token instruction that requests a complete technical report is still primarily an output workload. Prompt length and cost direction are related, but they are not the same thing.

The metrics that matter are average output tokens per request, output cost per request, and output cost per successful task. Group them by endpoint, model, prompt version, and response format. An org-wide average can look respectable while one code route emits complete files, one support route writes five-paragraph greetings, and one JSON endpoint produces an essay inside an explanation field nobody reads. Machines discover bureaucracy too, given enough schema properties.

Reasoning tokens are generated output you never read

Reasoning-capable models generate internal tokens before the visible answer. In OpenAI response usage, output_tokens_details.reasoning_tokens identifies them within billed output. Anthropic exposes billed internal reasoning through usage.output_tokens_details.thinking_tokens on supported thinking models. In both cases the visible response can be far shorter than the generated-token total you are charged for, and the response limit counts both.

That does not make reasoning waste by definition — a hard debugging task may justify internal work if it avoids retries or lands a correct patch. The mistake is enabling expensive reasoning uniformly for classification, formatting, extraction, or deterministic transforms without measuring whether it changes the result. Track reasoning or thinking tokens when exposed, then compare them against task accuracy, latency, retries, and human corrections. Invisible computation should still get a performance review.

Worked example: a file-rewrite endpoint

Assume a coding endpoint handles 10,000 requests/day for 30 days: 2,000-token prompt, 700-token complete file returned. Monthly input is 10,000 × 30 × 2,000 = 600 MTok; output is 10,000 × 30 × 700 = 210 MTok. At Claude Sonnet 5’s $2/$10 (check the current rate on the board): input ~$1,200/mo, output ~$2,100/mo. Output is 64% of the bill on a “small edit” feature.

Now swap full files for unified diffs averaging 120 output tokens for the same validated edits. New output is 10,000 × 30 × 120 = 36 MTok = ~$360/mo — a ~$1,740/month reduction, all of it output. Put both variants through the Burnmeter with your own token counts. The counts here are assumptions; the arithmetic is the part your logs should replace.

“Rewrite the whole file” regenerates unchanged code

When a model returns a complete file, every unchanged import, type, comment, test helper, and function body is generated again. Your invoice has no sentimental exemption for the 94% of lines that already existed. For small edits, request a unified diff, structured patch, or tool-based edit. A useful instruction: Return only a unified diff. Include enough context to apply it, no unchanged file content, and no explanation unless the patch cannot be applied safely.

Then validate mechanically — git apply --check, a parser, a formatter, a type checker, the test suite. A shorter response that will not apply is not efficient; it is compact failure. Full-file output stays reasonable when the consumer cannot apply patches, the file is genuinely being replaced, or a reviewer needs the surrounding code. The lever is not “always use diffs.” It is “stop paying to regenerate unchanged bytes without noticing.”

Budget the response at three layers

First, constrain the task in the prompt: ask for three findings, not “a comprehensive analysis”; for failing test names and patches, not a tutorial on testing. Second, constrain the structure — JSON Schema, enumerated labels, bounded arrays, a patch format. Third, set the provider’s output-token limit so a malformed request cannot produce several thousand tokens of syntactically valid regret. Do not treat the ceiling as a target; a generous limit does not force usage, but it lets verbose failure modes become billable literature.

Watch truncation and retry rates after tightening. If a 400-token cap makes 12% of requests retry with a larger budget, the apparent saving is just a second billable call. A practical system starts bounded, detects incomplete output, and escalates selectively. Run representative traces through the usage auditor and inspect endpoints with high output per successful result, not merely high output per request. Success belongs in the denominator, because cheap nonsense is still nonsense.

When output is not the real culprit

Output optimization will not rescue a workload dominated by enormous prompts, repeated conversation history, broad retrieval, cache misses, tool definitions, or agent loops. If output is a small share of reconstructed spend, swapping full files for diffs improves latency and reviewability while barely moving the invoice — useful, but not a major cost cut.

Check the usage export before choosing the remedy. Compare uncached input cost, cached-input cost, output cost, request count, retries, and separately billed tools. Also check whether shorter output degrades quality enough to create human cleanup. The correct conclusion may be that generated tokens are expensive but not numerous in your workload. Engineering would be easier if every invoice had one villain, but distributed systems have standards to maintain.

Frequently asked questions