Four kinds of software measure parts of AI coding ROI, and none of them measures all of it. The AI tools' own admin dashboards (GitHub Copilot, Cursor, Claude Code, Codex) report adoption and accepted code for their own tool. Engineering intelligence platforms such as DX, Jellyfish, LinearB, Swarmia and Faros AI connect usage across tools to delivery metrics, and some benchmark you against other companies. Session-level cost tools show what each agent session cost and what it was for. Work-outcome tools describe what shipped and how significant it was. None can prove an AI tool caused a business return. The question to ask of each is whether it can see the return side beyond counting artifacts.
Disclosure: Maestro AI, which publishes this blog, makes two of the products below, VibeCheck and Maestro. Vendor features are described from each vendor's own site as of November 4, 2026, and they change often.
Start with what "ROI" needs
The method in how to measure the ROI of AI coding tools needs three inputs: everything the AI costs (seats, metered usage and overage, from invoices), a baseline from before rollout, and the return, meaning what shipped and how significant it was.
The cost side is bookkeeping. The return side is where tools differ. Counting merged pull requests is the easy fallback, but AI makes PRs cheap to produce, so more of them is not more value; output was never the work. A rising PR count says activity went up. It doesn't say whether the extra capacity went to features or churn, or how much AI spend went into rework that never shipped.
So hold every category, ours included, to one test: can it see the return side beyond counting artifacts?
1. The AI tools' own dashboards and APIs
All four assistants below ship admin analytics.
- GitHub Copilot. GitHub's usage metrics cover active users, acceptance rate, lines of code and pull requests. In GitHub's words, "Pull request lifecycle metrics measure how Copilot activity relates to pull request outcomes and delivery flow." They're available as dashboards, APIs and an export.
- Cursor. Team analytics report agent edits, tab completions, messages, active users and cloud agent output. Its "AI Share of Committed Code shows the lines of code changed in commits to your repositories" and how much of that code came from Cursor.
- Claude Code. On Team and Enterprise plans, the analytics dashboard shows usage, and with the GitHub app connected it attributes merged PRs to Claude Code by matching session output against each PR's lines. Anthropic says plainly that "these metrics are deliberately conservative and represent an underestimate of Claude Code's actual impact." Contribution metrics are in public beta and "not currently available for API customers," who get usage and spend in the Console instead.
- OpenAI Codex. Workspace admins get Codex analytics to review adoption and activity, plus an Analytics API for loading aggregated reporting into a warehouse.
These are good at adoption and at how much code their own tool wrote; the PR views from Claude Code and Copilot get closer to delivery than an acceptance rate does. But each sees only its own tool, so three assistants means three definitions of "AI-assisted" and no shared denominator. None combines the other tools' usage with your invoices and salaries. On the return side they count lines and PRs a tool touched, which shows the tool was involved, not what the work was worth.
2. Engineering intelligence and analytics platforms with AI-impact features
These platforms already pulled data from git, CI and issue trackers, and have added AI usage and cost on top. Here's what each says about itself.
- DX says its AI analysis lets you "compare productivity between AI users and non-users across teams and over time" and "track current and potential AI-driven time savings, benchmarked against the industry."
- Jellyfish says it can "quantify AI spend, token usage, and impact on delivery, quality, and productivity across the SDLC," and "compare assistants, agents, and tools in a unified framework."
- LinearB builds its AI ROI view around cost per effective PR, which it defines this way: "Cost per effective PR divides your total engineering cost, people and AI combined, by your effective PRs." It pairs that with a rework-rate guardrail.
- Swarmia "connects with Claude Code, GitHub Copilot, Cursor, and Codex out of the box" and "combines AI compute costs with FTE effort" so you can see what a feature or initiative cost.
- Faros AI ties token spend to outcomes you define. On its ROI page, "ROI is expressed as cost per outcome instead of usage volume," and you can compare cohorts on spend and shipped outcomes.
If you need one view across several assistants, with history that predates your rollout, this is the category: cross-tool adoption, delivery and quality trends, cohort comparisons and, increasingly, all-in cost per unit of work. Their public pages describe git, CI, ticket and vendor usage data; ask each whether it can show what happened inside a single agent session, such as one that looped on a failing test. On the return side, ask each what its unit of work is and how it tells a significant change from a trivial one.
3. Session-level agent cost tools
These tools look inside agent sessions: what each cost, what it was for and how it went. That answers what a monthly total can't, like why spend jumped or how much went to rework. It's also the part you can't do by hand, because rework share lives in session transcripts and a few expensive sessions usually set the total. We list only our own product here; ask category 2 vendors whether they cover this.
VibeCheck (ours) is in this category. It captures Claude Code, Codex and Cursor sessions on each engineer's laptop and estimates each session's cost from its own tokens at published reference prices. It labels each session with the kind of work, rework and the times an engineer had to redirect the agent. On Pro it reports cost per authored PR. It doesn't capture GitHub Copilot or cloud agents. It shows what the spend went into: what each session was for and how it went. Cost per authored PR is an artifact count; read it as spend intensity.
4. Work-outcome tools
These read the work itself and describe what it accomplished, which aims them squarely at the return side. Again, only ours is listed.
Maestro (ours) is in this category. It reads tickets, reviews and code, scores every merged PR 0 to 5 for scope, complexity and effect, and categorizes work (features, bugs, maintenance), which gives you the capacity-allocation view a board asks for. A score describes how significant a change was, not what it earned the business or anything about a person's quality. For the cost half you still need invoices or a cost tool.
The categories side by side
| Category | What it measures | Blind spots | Examples |
|---|---|---|---|
| AI tools' own dashboards | Adoption, activity, accepted or attributed code for that one tool | Other tools, your baseline, your salaries, whether the work mattered | GitHub Copilot metrics, Cursor team analytics, Claude Code analytics, Codex analytics |
| Engineering intelligence platforms | Cross-tool adoption, delivery and quality trends, cohort comparisons, cost per unit of work; some offer industry benchmarks | Session-level detail varies, so ask; significance depends on how each defines a unit of work | DX, Jellyfish, LinearB, Swarmia, Faros AI |
| Session-level cost tools | Estimated cost per session, kind of work, rework, model choice, corrections | Invoice totals, Copilot, what shipped and how much it mattered, benchmarks | VibeCheck (ours) |
| Work-outcome tools | What shipped, its significance, capacity by kind of work | Business return, benchmarks | Maestro (ours) |
Start with the dashboards you already have. Then add whichever tool answers the question you still can't: cross-tool history and peer benchmarks, what happened inside sessions, or what shipped and how much it mattered.
No tool can attribute ROI, so be skeptical of a single number
Every product above can show that something changed. None can show the AI tool caused it; the rollout quarter might also be the quarter you reorganized two teams. Good software gives you a clean comparison: the same team before and after, or similar teams with and without heavy AI use.
Treat any single ROI figure as a model and ask to see its inputs. Take a fictional company, Fenwick Atlas, with 40 engineers and $180,000 a year in AI coding spend. Suppose a survey says each engineer saves 4 hours a week, and finance uses a loaded rate of $100 an hour over 46 working weeks. That prices the saved time at 40 × 4 × $100 × 46 = $736,000 a year against $180,000 of spend, and any ROI percentage computed from it rests on the survey.
Now assume 2 hours a week instead of 4: 40 × 2 × $100 × 46 = $368,000. Same made-up company, same spend, one assumption halved, and the "return" halves with it. Neither figure says whether those hours became shipped work, and hours priced at a loaded rate are a headcount equivalent, not business value.
A break-even is easier to defend. At the same rate and a 40-hour week, Fenwick's engineering time costs 40 × 40 × 46 × $100 = $7,360,000 a year. The AI spend pays for itself if significant shipped work rises by $180,000 ÷ $7,360,000, about 2.4%. The board question becomes checkable: did the work that mattered grow by more than that? Cost per merged PR tracks spend intensity alongside it, as long as nobody mistakes the PR count for the return.
Benchmarking against comparable companies
If the board wants to know how you compare with peers, only some vendors can help, and you should know what their pool contains.
- DX offers industry benchmarking and says you can "select from tailored benchmark segments by organization size, industry, and geography," or "benchmark yourself against a custom, individually-selected pool of companies."
- Jellyfish says on its homepage that it connects AI usage, cost and capacity to outcomes "benchmarked against 1,000+ engineering orgs."
- LinearB says its engineering metrics benchmark your organization "against 8.1 million+ pull requests."
Use external benchmarks as context, not targets. A peer's cycle time says little about yours if they ship a mobile app and you ship firmware.
Questions to ask any vendor
- Which AI tools do you capture, and what counts as AI-assisted? Find out what's missing (Copilot, cloud agents, a provider on Bedrock).
- Is cost from invoices or estimated? Estimates work for internal comparisons; totals for finance should tie to invoices.
- Can I get a baseline from before rollout? If history starts the day you install, so does your before/after.
- How do you see return beyond counting PRs? How do you tell a significant change from a trivial one, and what happens to rework and reverts?
- What's in your ROI number, and can I change the assumptions?
- For benchmarks: how big is my comparison pool, and who's in it?
Where VibeCheck and Maestro fit
VibeCheck shows what every Claude Code, Codex and Cursor session cost, what kind of work it was and how much rework it carried. Pro adds spend by team and cost per authored PR. Costs are estimates; take totals from your invoices. Free for up to five engineers: get started for free.
Maestro covers what shipped: every merged PR scored for significance, capacity by kind of work, written up weekly with links to the work. Neither product gives you an ROI percentage or cross-company benchmarks. Book a walkthrough, or for a larger rollout, talk to us.