Every engineering leader knows the DORA metrics: deployment frequency, lead time for changes, time to restore service, and change failure rate. They've become the de facto standard for measuring engineering effectiveness.
But here's the uncomfortable truth: DORA metrics measure activity, not impact.
The Promise of DORA
The DORA research gave us something valuable—a framework for thinking about software delivery performance. The four key metrics promised to help teams improve their velocity and reliability:
- Deployment Frequency: How often you ship
- Lead Time for Changes: How fast code goes from commit to production
- Time to Restore Service: How quickly you recover from incidents
- Change Failure Rate: How often deployments cause failures
On paper, these make sense. Faster deployments mean faster iteration. Quick recovery means better reliability. Lower failure rates mean higher quality.
The Reality Check
But in practice, optimizing for DORA metrics creates perverse incentives:
1. Gaming the System
Teams deploy trivial changes to boost deployment frequency. They split meaningful work into tiny commits to improve lead time. They lower the bar for what counts as an "incident" to game the change failure rate.
The metrics go up. The actual value delivered stays flat—or worse, decreases.
2. Missing the Forest for the Trees
DORA tells you how fast you're moving. It doesn't tell you where you're going or why it matters.
A team can have perfect DORA scores while:
- Building features nobody uses
- Accumulating technical debt that will cripple future work
- Burning out their best engineers with meaningless velocity targets
3. The Context Problem
DORA metrics treat all changes as equal. But they're not:
- A one-line config change ≠ A complex algorithmic improvement
- A cosmetic UI tweak ≠ Core business logic that drives revenue
- A quick fix ≠ Thoughtful refactoring that prevents future incidents
Without context, you're just measuring motion, not progress.
What Should You Measure Instead?
The answer isn't to abandon metrics. It's to measure impact, not activity.
At Maestro, we've built our platform around three principles:
1. Understand the Work
Use AI to read and understand code changes. Not just "how many lines" but "what does this actually do and why does it matter?"
Our Impact Score (0-5) captures:
- Technical complexity and risk
- Business value alignment
- Long-term maintainability
2. Recognize the People
Measure the quality of code reviews, not just the speed. Identify who's mentoring junior devs, who's tackling the hard problems, who's preventing future incidents.
Traditional metrics miss your heroes—the senior engineer who reviews thoughtfully, the architect who prevents bad decisions, the developer who documents complex systems.
3. Tell the Story
Numbers without narratives are just noise. Every metric needs context:
- Why did velocity drop? (Maybe the team is tackling technical debt)
- Why did incidents increase? (Maybe you're shipping ambitious features)
- Why is one developer "less productive"? (Maybe they're mentoring the team)
The Path Forward
DORA metrics aren't worthless—they're incomplete. Use them as health checks, not success criteria.
But to truly lead engineering teams in the AI era, you need intelligence that goes deeper:
- What's the actual impact of your team's work?
- Who's driving your products forward (beyond commit counts)?
- Where are the hidden bottlenecks and risks?
Stop measuring how fast your team moves. Start measuring whether they're moving in the right direction.
Want to see what impact-based engineering intelligence looks like? Book a demo to learn how Maestro helps engineering leaders measure what actually matters.
