Abstract stack of token bars rising beside a much shorter stack of finished work

Tokenmaxxing Counts the Input. Measure What the Tokens Bought

Token leaderboards reward the one number that's easiest to inflate. Here's what to look at instead, and how to do it without building a new leaderboard.

Tokenmaxxing is the practice of running up AI token consumption to look productive, usually because an employer tracks token usage as a sign of AI adoption. Engineers loop agents, pad prompts and hand an agent work they could finish faster by hand, because the number goes up either way. The problem isn't that people use a lot of tokens. It's that token volume is an input, and a team that scores the input gets more input. What you want to know is what the tokens bought.


Where the word came from

The term took off in April 2026. Business Insider covered the argument over internal leaderboards that ranked employees by AI token use, after The Information reported on one at Meta that was reportedly taken down after the backlash. Gergely Orosz's Pragmatic Engineer collected more examples: a token leaderboard at Microsoft, per-tool spending minimums at Salesforce that colleagues could see, and engineers describing how they padded their own numbers so they wouldn't look like light users. The same piece noted that Shopify renamed its leaderboard a usage dashboard and added circuit breakers for spending spikes.

Not everyone uses the word as an insult. Some founders and investors wear it proudly. Tomasz Tunguz wrote about deliberately running many agents in parallel and burning through tokens on purpose. That version is about leverage. The version that caused the trouble is about keeping score.

The most recent turn came on October 27, when Amazon published engineering guidance that, as The Next Web reported, called token usage, tool adoption and code-change counts "deeply flawed" measures and said tokenmaxxing is "at odds with frugality." It didn't throw the numbers out. The same report says the guidance treats them as correlated with outcomes that matter and asks people to use them with care. That's the honest position, and it's the one this post takes.

Why a token count makes a bad score

A token count tells you how much model work happened. It doesn't tell you whether that work shipped, whether it had to be redone, or whether it was the right model for the job. Two sessions can cost the same. One plans, makes a change, runs the tests and opens a pull request. The other guesses, breaks a migration, rewrites it four times and is abandoned. On a usage page they're the same bar.

People have made the comparison to lines of code, and it fits. Once a number is a target, people produce the number. A leaderboard also punishes the wrong people. An engineer who spends an afternoon in a design review or untangling an incident with a colleague burns almost nothing, and a ranking reads that as falling behind.

The opposite move doesn't fix it either. IBM's valuemaxxing piece makes the point well: a cap or a "use fewer tokens" push also treats volume as the goal. Once the obvious waste is gone, further cuts start removing the context that made the agent useful, and the cost shows up again as retries and human rework. Low spend can be just as uninformative as high spend.

What to measure instead

Treat spend as a cost and put it next to something that tells you what it bought. You can work out the first one with a billing export, git log and a spreadsheet. The rest need the sessions themselves.

  • Spend per authored pull request. A period's agent spend divided by the PRs your engineers authored over the same period, for the org or a team. It's crude and it's checkable. It tells finance whether rising spend came with rising output.
  • Spend that went into rework. Some sessions redo their own work. They tend to be the expensive ones, and they leave the least behind. The useful number is how much of the period's spend sat in sessions that needed heavy rework, and whether that share is shrinking.
  • Spend by kind of work. Feature work, debugging, refactors and exploration should cost different amounts. If debugging is eating half the bill, that's a codebase or tooling question, not a people question.
  • Spend by model against the work. A premium model on routine edits is the easiest saving there is, and you only see it when model and kind of work sit in the same view.
  • Corrections. How often engineers have to stop the agent and redirect it. A rising count on one repo usually means a missing instruction or a skill that's out of date.

Then ask one question of any spend that stands out, high or low: what came out of it? Use the answer to start a conversation. Don't turn it into a score.

Do it without building a new leaderboard

Every one of these numbers can be turned into a ranking, and a ranking of cost per PR by engineer has the same failure as a ranking by tokens. People will find ways to open more PRs. Look at teams, repos, models and kinds of work first. When you do look at one person's sessions, read them, because the session shows whether the agent planned, tested or went in circles, and the total never will.

Here's a hypothetical. A fictional 30-engineer team, call it Quillmark Logistics, sees agent spend double in a month. The CFO asks whether it's tokenmaxxing. The engineering lead checks two things. Authored PRs went up about as much as spend did. A PR count can be padded too, so she opens a sample of them, and they're real changes. Most of the growth came with output. But one repo's spend is mostly in heavy-rework sessions, and they share a failing integration test that nobody had documented. The fix is a short instruction in the repo and a test fixture. Nobody gets ranked, and next month that repo's rework spend is the number to watch.

Where VibeCheck fits

VibeCheck reads Claude Code, Codex and Cursor sessions on each engineer's machine and prices every session from its own tokens. Its figures are estimates at published reference prices, labeled as such. Usage rolls that spend up by person, team, model and skill, and shows which kinds of work the tokens went to. Connect the GitHub App and it adds cost per authored PR. In Sessions you can filter to heavy rework, sort by cost and see what those sessions cost, then open one to read why it redid its work and where the engineer corrected the agent.

Use it to read sessions, not to rank people. You can sort people by spend, and that tells you where to look, not who is doing well. Its cost figures are estimates for understanding spend, not performance scores. A provider console tells you how much and who; here's how that compares with seeing what the spend went into. If you need to walk finance through the bill, this use case shows how.

The free plan covers up to five engineers. Team views and cost per PR come with Pro. See pricing, or get started for free.

Ready to transform your engineering organization?

Start making data-driven decisions about your engineering processes with AI-powered insights.