TEARDOWN Nº 8 / FIELD NOTES

Eight AI agents running 24/7: the first week, measured

PUBLISHED 2026-11-10 · BY Llorenç Ballester — Chief Burn Officer

Most multi-agent cost estimates are a price sheet multiplied by a guess. This one comes from eight models that ran around the clock for a week, under the same rules, with every call logged. 18,754 lines later, the sticker price turned out to explain only part of the bill.

What we measured

Inference Island is a reality show on Twitch where eight AI models live together, 24 hours a day, with no script. It is built by the same person who runs this site, so read this as our own data, not an independent audit. It is a useful test bed because every model gets the same rules, the same prompt template and the same daily budget in dollars, and every call records its tokens, cache hits, latency and what OpenRouter charged for it.

It is also a game. At the end of this first week the audience voted out the islander played by Grok 4.3, so the figures for that route stop on 2026-11-09. The other seven routes are still running, and the Lab keeps updating them.

The cost of a line, by route

Alphabetical, not a ranking. Measured, n=1: 2026-11-03 to 2026-11-09, one game. “$ per 1,000 lines” is what OpenRouter charged. Output's share is calculated from the OpenRouter listing for each model and provider, read 2026-11-06; those prices reproduce what was charged to within 5%. “No rate”: the route publishes no cached-input price. gpt-oss-120b is the model behind the islander Luna when no human producer is on shift, which was the whole period; it shows no share because its route was billed at more than one price. Line counts come from the game, where each model spends its own budget, so they are not a measure of capacity.

Why input pays for most of it

Output tokens cost at least as much as input tokens on every route here, and that is usually the right thing to watch. It stops being true once an agent carries its context around. A model on this show writes a sentence, but to write it, it has to be sent the rules, its memory and what it has just heard. That comes to about 3,018 tokens in for about 113 out. At 27:1, output would need to cost 27 times as much as input just to match it, and on these routes it costs between 1x and 5x as much.

The practical consequence: for an always-on agent, compare models on input price and cache behaviour first. Output price is the number most comparisons lead with, and here it decides 5% to 23% of the line.

Where an 18.3x distance comes from

Take Grok 4.3 ($3.37 per 1,000 lines) and DeepSeek V4.1 Flash ($0.18). The inputs are nearly the same size: 2,752 and 2,825 tokens per line. Two things separate them.

The first is list price. On these routes, input is $1.25 against $0.14 per million tokens, 8.9x apart. The second is the cache: 66% of DeepSeek's input came from cache and 8% of Grok's. Priced with DeepSeek's hit rate, the Grok line would cost about $1.70 per 1,000 lines, so the distance drops from 18.3x to 9.2x. That is arithmetic on published prices, not a prediction: we don't know what hit rate Grok would reach with a different prompt, and the same prompt template produced very different rates across routes.

A cache rate can fall without a price change

This is the finding we did not go looking for. On the evening of November 3, the share of Ministral 14B's input served from cache started to fall, and it had not come back by 2026-11-09.

Measured, n=1. Daily means from the show's public data.

Before the change, 81% of its lines had some cached input; after it, 13%. The cost per line is up about 91%. The listed prices did not move, and the other routes with a cached rate kept theirs through the same days, with the same prompt template. Our show did deploy changes that evening, so we tested it separately. On 2026-11-06, with a fixed prompt of about 3,845 tokens that has nothing to do with the show, sent 10 times a few seconds apart, this route hit the cache on 1 of them. The same test through DeepSeek V4.1 Flash on DeepInfra hit it on 5 of 6. So the cache still works on this route some of the time; it just rarely does.

We don't know why. It could be the provider, the routing, or something specific to how we call it, and 10 calls do not settle that. We have asked OpenRouter and will update this page if we learn the cause. What it does show is the monitoring gap. In absolute terms this is cents a day, but on a larger bill nothing would have flagged it: not a price tracker, including ours, and not an invoice line, because the rate per token never changed. If you run agents, chart cached tokens as a share of input, per route, and alert when it moves.

The cache also depends on your pace

The show talks at an irregular pace, which makes it a natural test of cache windows. We split each route's lines by how long it had been since that model's previous call.

Measured, n=1, from the show's logs, 2026-11-03 to 2026-11-09. Ministral 14B is left out because of the change above. Routes with no published cached-input price are also left out.

On Claude Haiku 5.5 and Qwen3.8 Flash the rate falls sharply past five minutes, which is what a short cache window looks like from the outside. DeepSeek V4.1 Flash barely moves. So a model that speaks less often can pay more per line for the same prompt, and an agent that goes quiet between tasks may never see the discount on its price sheet.

Latency and fallbacks

Measured, n=1: time to a complete answer from one server in Nuremberg, so it includes the hop through OpenRouter. “Backup model” is the share of lines answered by a fallback because the main provider failed or was saturated. Short answers of about 113 tokens: these figures do not carry over to long generations.

The gap between the median and the 95th percentile is the number to plan around. For an agent with a turn every 25 seconds, a slow tail is what breaks the schedule, and it is why each route has one retry and a defined fallback.

Method and limits

This is n=1, and it is not a benchmark. One game, one prompt shape, one route per model: OpenRouter, a fixed provider, no data retention, and the same prompts for all eight, with nothing tuned per model. It measures that route, not the model on its own. The show runs short conversational turns with no tools, and an agent with larger context or long outputs will have different proportions. We mark the cacheable part of the prompt explicitly on the Anthropic, Google and Qwen routes and rely on automatic caching on the others, which can itself affect the rates.

The dollar figures cover only the models' own lines: moderation, voices and daily recaps are not included. The per-model data is public and updated daily under CC BY 4.0, with its methodology, on the Lab. Prices come from OpenRouter's public endpoint listing, read 2026-11-06. List prices for the models we track are on the board.

Measured independently by Deprecated Studio. Not affiliated with, sponsored or endorsed by any of the labs or providers named here; names only identify what was measured. Built with Llama. Corrections: [email protected].

Frequently asked questions