I sell a code review that a model performs. Twelve of them have now run against real repositories, and the invoice is public here because almost nobody publishes these numbers and the guesses I see online are wrong in both directions.
Twelve scans. $30.27. An average of $2.52 each. Each one sends roughly 140,000 tokens of code and asks about 19 separate questions of it.
Where the money actually goes
The naive version of this costs a fortune: 19 questions × 140,000 tokens is 2.6 million tokens of input per scan, at full price, every time.
It does not work that way, because of one number: a cached prompt prefix is billed at a tenth of the normal input rate. So you send the code once, cache it, and fire every question at the same cached block.
Across those twelve scans, 29,910,071 tokens were read from cache — about 2.5 million per scan. That is the same 140,000 tokens of your code being re-read eighteen or nineteen times, at a tenth of the price each time, instead of being uploaded afresh for every question.
The mistake that costs more than not caching at all
Here is the part that surprised me, measured on one repository at 150,000 tokens and 22 calls:
| Approach | Cost |
|---|---|
| Cached, calls in sequence | $2.63 |
| Batched, cache warmed first with a 1-hour TTL | $1.96 |
| No caching at all | $5.51 |
| Batched with no warm-up | $6.75 |
Batching without warming the cache first is worse than not caching. That is not intuitive and it cost me real money to learn.
The reason: a cache entry only becomes readable once the first response has started streaming. Fire twenty requests simultaneously and none of them can read what the others are writing — so every one of them pays the write price, and you have paid to populate the same cache twenty times over.
The fix is a throwaway request with max_tokens: 0 before the
batch, and a one-hour TTL rather than the five-minute default. Five minutes
expires partway through a long batch, and the reads silently become writes.
Nothing errors. The code looks identical. The bill quadruples.
What this means if you are building on top of a model
Three things I would tell anyone doing something similar:
Put the stable thing first. Cache the document and vary the question, never the reverse. Caching the questions and streaming the document past them roughly triples the bill for the same work.
Measure the cache read count, not the invoice. A broken cache does not fail — same output, same behaviour, four times the cost. The only symptom is a number nobody looks at. I record cache reads on every scan and alert when one comes back at zero.
Cap the input before you spend anything. Token counts are checked before a scan starts. Without that ceiling, one enormous repository is an unbounded bill, and it is the only real attack on a fixed-price product.
The honest limits of these numbers
Twelve scans is a small sample and every one was 136,000–147,000 tokens, because the input is capped. A different workload — shorter documents, more questions, a different model — changes everything above except the cache mechanics.
What generalises is the shape: the cost is dominated by how many times you re-send the same context, and the difference between doing that well and doing it naively is not a few percent. It is 2 to 3×.