TEARDOWN Nº 4 / THE TOKENIZER TAX

The tokenizer tax: a price rise with no price change

PUBLISHED 2026-08-13 · BY Llorenç Ballester — Chief Burn Officer

Every price tracker on the internet, this one included, watches the same number: dollars per million tokens. That number can sit perfectly still while your bill for the same work goes up, because the other half of the equation — how many tokens your text becomes — is not on any pricing page. Anthropic has published exactly that change, and it is roughly 30%.

The fact, and whose it is

From Anthropic's own pricing documentation, read on 2026-08-13: Claude 4.7 and later models use a newer tokenizer that “produces approximately 30% more tokens for the same text”, and “the exact increase depends on the content and workload shape”. Claude Sonnet 4.6 and earlier use the previous one.

That is their figure, approximate and content-dependent, and we have not measured it — there is no Claude tokenizer we can run in a browser to check. Everything below propagates their stated number through our own price board and is labelled as a sensitivity, never as a measurement. If the real figure for your content is 15% or 45%, the table further down shows you that too.

Which side of the line each model sits on

Classified by Anthropic's published rule (4.7 and later). Note the trap in the model ids: Claude Haiku 4.5 is version 4.5, below the cutoff — its trailing “5” is a minor version, not a generation. We got that wrong in a first pass and a test caught it. The 12 non-Anthropic models on the board are left unclassified: no vendor rule is published for them and we are not inventing one.

What it does to a price comparison

Take the two Anthropic models on opposite sides of the line. On the pricing board, Claude Sonnet 5 costs 2.00x Claude Haiku 4.5 per million input tokens. That ratio is correct — and it answers a question nobody asks. What you actually want to know is what the same document, the same prompt, the same job costs on each.

This generalises past Anthropic. Any comparison of dollars per token between two models on different tokenizers is comparing two different units and calling them the same. Our own head-to-head pages do exactly that, as does every pricing table we know of. It is not wrong so much as incomplete, and the size of what it omits is the number above.

If the real figure isn't 30%

Anthropic says “approximately” and makes the figure content-dependent, so a single number would be false precision. For Claude Sonnet 5 on a 1,000-token prompt with a 300-token reply:

Cost moves one-for-one with token count, because the rate is fixed. That is the whole mechanism: nothing clever happens, which is precisely why nothing catches it.

What this does to our own numbers

The uncomfortable part, and the reason this teardown exists rather than a news post. Our calculator counts tokens with o200k — OpenAI's tokenizer — for every model on the board, because it is the one that runs in your browser without shipping your text anywhere. We have always said so, alongside a stated margin of roughly ±10% for non-OpenAI vendors.

A ~30% shift between two generations of a single vendor is comfortably larger than that margin. So the caveat was too optimistic for Claude 4.7+ models, and we have rewritten it rather than leave a number on the page we no longer believe. The count you get for a Claude model is an approximation whose error we cannot currently bound from the browser — that sentence is less satisfying than “±10%” and it has the advantage of being true.

The board itself is unaffected: prices per token are cross-checked daily against the source and were never in question here. What this changes is the translation from your text to tokens, which is the step before the price applies — and the step nobody audits.

What to do about it

1. Trust the provider's own usage numbers over any estimate, including ours. The usage block in the API response is counted by the tokenizer that actually billed you. Every third-party estimate, this site included, is a model of that.
2. Re-baseline after a model upgrade, not just after a price change. Moving from a 4.6-era model to a 4.7+ one is a cost event even when the rate card is identical.
3. Compare jobs, not tokens. Price the same text through both candidates using each one's reported usage — which is the same argument our canonical workloads make, and this is the sharpest example of why.

Method and limits

The ~30% figure, the 4.7 cutoff and the rule for which models fall either side are Anthropic's, quoted from their pricing documentation on 2026-08-13. We did not measure the inflation and do not claim to; every figure here is that stated number propagated through prices from our board, recomputed at build time. Non-Anthropic models are deliberately left unclassified because no equivalent rule is published for them — an absence of evidence, which we are not going to report as evidence of absence. Full rules at /methodology.

Frequently asked questions