Back in June, Gartner put out a prediction that made the rounds fast: by 2028, they say, the cost of AI coding assistance will actually exceed the average developer's salary. The Register ran with it. CIO ran with it. And my inbox promptly filled up with a dozen variations of the same nervous question: "are we insane for spending this much on tokens?"

Maybe! But before anybody hits the panic button, I want to walk you through what I found when I actually sat down and audited where our tokens were going, because I'll be honest, the answer embarrassed me, and I strongly suspect it would embarrass most teams too. Our bill wasn't enormous because the models are expensive. Our bill was enormous because we were feeding them like a broken slot machine.

I've been the guy tuning JVM heap flags to shave a few points off a hosting bill. I've been the guy who finally tracked down the $40k-a-month CloudWatch line item that nobody in the entire org could explain. And I'm telling you, token spend in 2026 has the exact same shape as cloud spend did back in 2015: a brand-new metered resource, zero organizational muscle for actually managing it, wildly different usage from team to team, and a whole vendor ecosystem more than happy to sell you the panic. The good news is the playbook that eventually fixed the cloud bill fixes this one too. Measure. Attribute. THEN fix the workflow. The pricing page is the very last thing you touch, not the first.

Where do the tokens actually go?

When people picture their AI bill, they picture the conversation: prompt goes in, code comes out. But when we finally broke our real usage down, the conversation itself turned out to be a minority of the spend. The bill actually has at least four completely distinct components, and here's the important part, they each respond to a totally different fix.

The context you're shipping on every single prompt. System prompts, project instructions, memory files, hook-injected preambles, whatever else your tooling quietly prepends to everything. Ours had crept up to several thousand tokens, and here's the thing, nobody ever actually decided that. It just accreted, the exact same way a CI pipeline accretes steps, one perfectly reasonable addition at a time. Now multiply that by every prompt, from every developer, every day, and it turns out to be our single largest controllable line item by a mile. We cut it roughly in HALF in one afternoon, just by making the injection conditional: worktree context only when a worktree is actually bound, the review rubric only during an actual review, and so on down the list.

Tool schemas, loaded once and cached (right up until they aren't). Every single MCP server you connect ships its tool definitions straight into the context. Individually it's cheap, sure, because caching amortizes it away. But collectively? We found sessions hauling around 30-plus tools when the median session actually used four. And cached is not the same as free, either: caches have TTLs, and an idle terminal that drifts past the cache window just re-pays the entire load on its very next prompt. Disconnecting the servers a given session doesn't need was pure, clean savings, with genuinely zero loss of capability for that session.

The background and automated runs. The QA review that fires the second a task completes. The security audit. The summarizer. The scheduled job somebody set up back in March and then completely forgot about. Every one of these is an agent run that the user never actually typed. These are your "per run" costs, and they dwarf the per-prompt costs the way a full build dwarfs a single keystroke. One of ours, it turned out, was re-reviewing an entire diff on every single iteration of a loop. And nobody noticed, because nobody was attributing spend down at that level of granularity. Which is really the whole lesson right there: you simply cannot manage what lands in one big undifferentiated "usage" number.

The retry loops and the thrash. The agent that reads your ENTIRE repo because nobody told it where to actually look. The one that re-runs the same failing test fifteen times without ever once changing its hypothesis. The stuck loop that quietly burns through a whole session's budget overnight producing a branch that not a single soul asked for. This is the purest waste category of them all, and it is entirely, top to bottom, a supervision problem: iteration caps, a hard budget ceiling per run, and a human actually glancing at the thing every so often.

And notice what is conspicuously ABSENT from that entire fix list so far: switching providers, downgrading models, or standing in the street yelling at the pricing page.

Context engineering IS cost engineering

The discourse likes to treat "context engineering" as a quality discipline, which, to be fair, it absolutely is. Feed the model exactly what's relevant and you get sharper answers. But what gets far less airtime is that it is the very same lever as cost, and it pulls in the very same direction. The bloated context that's quietly costing you money is also actively degrading your output, because the relevance is diluted and the poor model is off dutifully attending to your stale instructions from two features ago. So every token you trim for cost reasons also improves the answer. And listen, there are not many optimizations in this entire business where the tradeoff points the same direction twice. When you find one, you take it.

Scoping is the other half of it. An agent pointed squarely at src/billing/ with a frozen interface contract does not need to go read your whole monorepo, does not spend the tokens doing so, and (bonus) physically cannot get creative in files it was never allowed to see. Cheaper, better, AND safer all turn out to be the same single move. And effort tiers matter here too: we were running maximum reasoning effort on rename-this-function tasks, which is, let's be honest, hiring a principal engineer to come change a lightbulb. Route the trivial work to the cheap, fast models, and save the expensive deliberation for the problems that are genuinely, actually hard.

When you should just happily eat the cost

Frugality has its own failure mode, and I've watched good teams fall right into it: treating tokens as the scarce resource, when the truly scarce resource is, still, always, senior engineering attention.

A $50 overnight agent run that lands you a viable first draft of a migration you'd otherwise have to schedule two full senior days for is not an expense. It's the single cheapest labor you have ever purchased in your life, by an entire order of magnitude, and the correct response to it is to go run MORE of them, not fewer. Gartner's whole framing (the ominous "it'll cost more than a developer!") quietly smuggles in the assumption that the comparison is cost versus cost. It isn't. It's cost versus output. If an agent fleet that costs you one salary reliably produces meaningfully more than one additional salary would have, then that prediction isn't a warning at all. It's a shopping list.

And here's the genuinely uncomfortable truth buried inside the whole cost panic: both things are true at the exact same time. Most teams really are wasting a huge fraction of their spend on the four categories up above. AND most teams are simultaneously UNDERspending on the handful of runs that actually matter. The waste and the underinvestment are living together in the very same bill, which is precisely why that one big undifferentiated number tells you absolutely nothing worth knowing.

The boring checklist that actually worked

Here's what genuinely moved our bill, ranked honestly by effort-to-savings:

  1. Made the per-prompt injection conditional instead of always-on. One afternoon of work, roughly halved the single biggest line item.
  2. Trimmed the MCP servers, per project, down to only what that project actually uses.
  3. Put iteration caps and a hard per-run budget ceiling on every automated loop.
  4. Attributed the spend per feature and per background job, so that forgotten March cron job finally had nowhere left to hide.
  5. Routed the low-stakes tasks to cheaper, faster models, and kept the expensive reasoning for the genuinely hard problems.
  6. THEN, and only then, went and looked at plans and pricing. By which point, funny enough, the urgency had completely evaporated.

(Full disclosure: visibility into exactly this stuff is a big part of why I built the tool this blog is attached to; it literally badges what each feature adds to your prompts and what each background run costs you, before you ever enable it. But I want to be crystal clear, every single item on that list up there is doable with a spreadsheet and one honest week.)

Your AI bill is trying to tell you something, the very same way your AWS bill was trying to tell you something back in 2015. It's just not actually saying "the vendor is too expensive." It's saying "you haven't looked yet."