Developer Productivity Insights from DORA Creator Nicole Forsgren: Frameworks, Metrics, and AI's Impact

DORA framework creator Nicole Forsgren reveals why PRs are incomplete metrics, how to build high-performing teams, and how AI is reshaping developer productivity measurement.

Dr. Nicole Forsgren, creator of DORA and SPACE frameworks, shares insider knowledge on measuring developer productivity, building platform teams, and navigating AI's transformation of software engineering.

Key Takeaways

• PRs and diffs are both "good signals and terrible signals" when used alone—they require context from satisfaction, performance, and collaboration metrics
• Only 20% of developer work is visible in traditional metrics, while senior engineers spend most time unblocking others, reviewing, and mentoring
• DORA's four metrics (deployment frequency, lead time, change failure rate, recovery time) focus solely on commit-to-production pipeline optimization
• SPACE framework expands measurement to include Satisfaction, Performance, Activity, Communication, and Efficiency across the entire development lifecycle
• Developer experience encompasses the entire lived experience of doing work daily, not just tooling or IDE functionality
• High-performing teams share psychological safety, curiosity about better approaches, and fast onboarding processes that predict overall productivity
• Onboarding time reveals systemic inefficiencies—companies with months-long onboarding often have fundamental tooling and documentation problems
• AI coding tools are changing developer work from pure coding to "driving and reviewing" with increased emphasis on trust and verification

Timeline Overview

• 00:00–02:03 — Intro: Nicole's background as DORA and SPACE creator, current work at Microsoft Research, and upcoming developer experience book
• 02:03–07:42 — PRs and Diffs and how to view them in the right context: Why traditional productivity metrics fail, senior engineer work patterns, and contextual measurement importance
• 07:42–10:26 — EngThrive at Microsoft: Microsoft's approach to developer productivity measurement using qualitative and quantitative data combinations
• 10:26–17:00 — The importance of having a holistic set of metrics in evaluating productivity: Performance management vs productivity optimization, Hawthorne effect, and SPACE framework components
• 17:00–23:57 — The four key metrics of DORA: Deployment frequency, lead time, change failure rate, recovery time, and their scope limitations
• 23:57–26:40 — The evolution of processes and tools since DORA, including SPACE: Expanding beyond commit-to-production to full developer experience measurement
• 26:40–30:44 — An explanation of developer experience — and ways to improve it: Lived experience definition, friction identification, and system boundary impacts
• 30:44–34:20 — Devex at startups vs. larger companies: Trade-offs between resources and bureaucracy, legacy system challenges, and infrastructure investment differences
• 34:20–39:05 — Why measuring developer productivity is so difficult: Invisible work nature, increasing system complexity, and leadership disconnection from developer pain
• 39:05–44:34 — How to make a case for platform teams: Combining data and stories, resource trade-off discussions, and tipping point acknowledgment
• 44:34–51:01 — Common characteristics of highly productive teams: Psychological safety, curiosity, purposeful work, and continuous learning approaches
• 51:01–52:49 — Brook's law and how faster onboarding might make it irrelevant: Fast onboarding enabling rapid team scaling without traditional productivity penalties
• 52:49–54:18 — Onboarding for internal transfers: Distinguishing between new environment onboarding vs familiar system transitions within organizations
• 54:18–58:36 — Shifting culture towards technology first: Changing work patterns to influence culture, tool improvements driving behavioral change
• 58:36–1:03:36 — How middle management can improve engineering culture: Listening tours, ally building, quick wins through hack days, and aligned incentives
• 1:03:36–1:06:42 — How AI tooling is impacting developer productivity: DORA metric stability, SPACE framework expansion, trust factors, and workflow transformation
• 1:06:42–1:08:40 — Potential use cases for AI: Release engineering, deployment decisions, risk assessment, and engineering lifecycle automation opportunities
• 1:08:40–1:15:30 — A case for experimenting with AI coding tools and how to maintain flow state: Different usage patterns, flow preservation, and workflow adaptation strategies
• 1:15:30–END — Rapid fire round: Coding vs research preferences, upcoming developer experience book, productivity tools, and book recommendations

Rethinking Developer Productivity Measurement

The traditional approach to measuring developer productivity through pull requests and code commits represents a fundamental misunderstanding of how software engineering work actually operates in modern development environments.

• PRs serve as incomplete proxies for actual productivity because they only capture visible outputs while ignoring the substantial invisible work that senior engineers perform daily
• Senior engineers produce fewer commits by design as their responsibilities shift toward unblocking teammates, architecture decisions, code reviews, mentoring, and cross-team collaboration activities
• Context determines metric value more than absolute numbers since high PR counts might indicate either productivity or system inefficiencies requiring excessive small changes
• Traditional economic productivity definitions fail in software where outputs can't be easily quantified like physical goods, and value creation often happens through prevention rather than production
• Constellation approaches provide better insights by combining activity metrics with satisfaction surveys, performance outcomes, and collaboration effectiveness measurements
• Hawthorne effect influences all measurement efforts as developers naturally optimize for whatever metrics leadership tracks, making single-metric approaches particularly dangerous

The shift from measuring individual output to understanding system effectiveness requires recognizing that software development is fundamentally a collaborative knowledge work activity where the most valuable contributions often remain invisible.

DORA Framework: Strengths and Strategic Limitations

DORA's four key metrics have provided the software industry with its first widely-adopted framework for measuring development pipeline effectiveness, while maintaining clear boundaries about what it does and doesn't measure.

• Deployment frequency measures pipeline capacity for how often teams can release changes to production environments, indicating system scalability and process efficiency
• Lead time tracks change delivery speed from commit to production deployment, revealing bottlenecks and friction points in the development workflow
• Change failure rate quantifies quality outcomes by measuring what percentage of production changes cause service degradation or require immediate fixes
• Recovery time evaluates incident response effectiveness showing how quickly teams can restore service when production issues occur
• Commit-to-production scope provides specific focus on the most engineerable part of the software development lifecycle where teams can optimize systematic processes
• Signal quality improves with proper implementation as teams identify blockers, friction points, and systematic inefficiencies that prevent faster, more reliable delivery

DORA's deliberate focus on the post-commit pipeline reflects the reality that "you should have opportunities to optimize all of that system" with predictable, low-variability outcomes.

SPACE Framework: Holistic Developer Experience Measurement

The evolution from DORA to SPACE represents a fundamental expansion in how organizations can understand and optimize the complete developer experience across all aspects of software engineering work.

• Satisfaction metrics capture subjective developer experience through surveys asking about tool satisfaction, work enjoyment, and perceived productivity rather than relying solely on objective measurements
• Performance encompasses quality and outcome measures including security results, stability metrics, customer value delivery, and business impact rather than just activity counts
• Activity tracks countable behaviors and events such as commits, PR reviews, meeting participation, and system interactions while avoiding the trap of using counts as sole productivity indicators
• Communication and collaboration measurement examines meeting patterns, API reliability, cross-team interactions, and knowledge sharing effectiveness across organizational boundaries
• Efficiency and flow analysis focuses on handoff delays, wait times, system bottlenecks, and the time required to complete end-to-end workflows
• Multiple category coverage strengthens insights by ensuring that at least three different SPACE dimensions are measured to avoid blind spots and provide comprehensive understanding

SPACE recognizes that "Dora is an instance, it's an instantiation of space" where deployment frequency represents activity, lead time shows efficiency, and failure metrics indicate performance.

Developer Experience: Beyond Tools to Lived Experience

Developer experience encompasses the complete daily reality of software engineering work, extending far beyond IDE functionality to include every system, process, and interaction that affects how developers accomplish their goals.

• Lived experience defines developer experience as the actual day-to-day reality of doing software engineering work, including both positive and negative interactions with tools, processes, and systems
• Friction identification reveals improvement opportunities by examining where developers encounter delays, confusion, broken systems, or unnecessary complexity that impedes productive work
• System boundary impacts often go unrecognized when changes to HR systems, expense processes, or compliance requirements create developer productivity drags that leadership doesn't associate with engineering efficiency
• Cognitive load considerations matter significantly as developers can only handle limited frustration before their creativity and problem-solving abilities become compromised
• Manual intervention requirements indicate optimization opportunities where automated preconditions and environment setup can eliminate repetitive developer tasks
• Security and compliance integration should minimize developer friction through automated provisioning rather than requiring manual approval processes for routine access needs

The goal is creating environments where developers can focus on creative problem-solving and innovation rather than fighting with broken systems and unnecessary process overhead.

Characteristics of High-Performing Engineering Teams

Research across diverse technology environments reveals consistent patterns among teams that consistently deliver high-quality software efficiently while maintaining sustainable development practices.

• Psychological safety enables risk-taking and learning by creating environments where team members feel comfortable asking questions, admitting mistakes, and proposing new approaches without fear of blame
• Curiosity about better approaches drives continuous improvement through openness to the possibility that current methods aren't optimal and willingness to investigate how other teams solve similar problems
• Purposeful work connection maintains motivation when team members understand how their projects contribute to broader organizational goals and customer value rather than working in isolation
• Fast onboarding processes indicate systematic efficiency as teams with quick new hire integration typically have excellent documentation, clear processes, and well-designed development environments
• Learning orientation embraces change by recognizing that solutions that were impossible or difficult five years ago may now be solved problems with available tools and techniques
• Question-asking culture accelerates knowledge transfer through regular discussions about project alignment, process improvements, and technical decisions rather than assuming understanding

The most productive teams treat onboarding time as a crucial indicator: "how long does it take a new hire to commit code" reveals underlying system health and process quality.

Making the Business Case for Platform Investment

Engineering leaders must present compelling arguments for platform team investment that balance technical needs with business realities while acknowledging resource constraints and tipping points.

• Data and stories combination provides compelling evidence by pairing quantitative measurements of developer time waste with concrete examples of daily friction that business leaders can understand
• Trade-off discussions acknowledge resource constraints by presenting realistic options for repurposing existing engineers versus accepting reduced feature velocity during platform improvements
• Time estimation exercises reveal hidden costs through simple spreadsheets tracking how long routine tasks take across teams, enabling back-of-napkin calculations of aggregate productivity loss
• Tipping point acknowledgment builds credibility by explicitly stating that platform investments have diminishing returns and won't solve every problem, demonstrating thoughtful resource allocation planning
• Leverage point identification focuses investment on specific bottlenecks that affect multiple teams and can't be solved by individual development groups working independently
• Ongoing reassessment commitments ensure that platform teams will be scaled down once systems reach "good enough" status rather than becoming permanent resource drains

Successful platform investment proposals recognize that "further investments won't deliver that much more progress" and commit to continuous evaluation of value delivery.

Cultural Transformation Through Work Pattern Changes

Research demonstrates that lasting organizational change happens more effectively by modifying how people do their daily work rather than attempting to change mindsets through communication and training alone.

• Work pattern changes drive cultural evolution as new tools and processes that improve daily experience naturally shift how teams think about software development and collaboration
• Leadership involvement through hands-on experience creates authentic understanding of developer pain points when executives spend time using the systems their teams work with daily
• Listening tours and ally building enable middle management to understand current challenges and build support networks before proposing systematic changes to established workflows
• Quick wins through hack days provide visible progress on persistent frustrations while demonstrating leadership commitment to developer experience improvements
• Aligned incentives reinforce cultural changes by ensuring that performance reviews and recognition systems value platform improvements and developer experience work alongside feature delivery
• Explicit commitment demonstration shows that investments in technical infrastructure and developer productivity receive the same priority and recognition as customer-facing feature development

The key insight is that changing how work gets done naturally influences how people think about work, rather than trying to change thinking first and hoping behavior follows.

AI's Transformation of Developer Productivity

Artificial intelligence coding tools are fundamentally reshaping software development workflows while creating new categories of measurement and optimization opportunities across the engineering lifecycle.

• DORA metrics remain relevant for pipeline measurement as deployment frequency, lead time, and quality metrics continue providing valuable insights into development system effectiveness regardless of code generation methods
• SPACE framework expands to include trust factors since developers now must evaluate AI-generated code quality and make decisions about when to accept, modify, or reject automated suggestions
• Workflow patterns shift from pure coding to review-heavy processes where developers spend more time examining and validating generated code rather than writing everything from scratch
• Trust becomes a critical new measurement dimension as teams need metrics for evaluating when AI suggestions are reliable versus when human oversight is essential for quality and security
• Release engineering opportunities multiply through AI-assisted risk assessment, deployment decision support, and automated analysis of complex system data for release readiness determination
• Flow state considerations require individual adaptation as some developers prefer inline completion while others find separate AI consultation preserves focus and concentration better

Early research shows that "seasoned developers right they have at least five years of experience" are seeing "over a 50% increase in unit tests getting passed" when using AI coding assistants effectively.

Common Questions

Q: Are PRs and diffs good metrics for measuring developer productivity?
A: They're both good signals and terrible signals when used alone—they need context from satisfaction, performance, and collaboration metrics.

Q: What makes DORA metrics useful despite their limitations?
A: They provide reliable indicators of development pipeline health from commit to production, the most engineerable part of software development.

Q: How should teams approach AI coding tool adoption?
A: Experiment with different workflows, find what works for individual developers, and be open to fundamentally different ways of working.

Q: What's the best way to make a case for platform team investment?
A: Combine quantitative data on time waste with concrete stories, acknowledge trade-offs, and commit to reassessing value delivery over time.

Q: How can middle management improve engineering culture?
A: Focus on changing work patterns through better tools and processes rather than trying to change mindsets through communication alone.

Conclusion

Nicole Forsgren's research reveals that effective developer productivity measurement requires moving beyond simplistic metrics toward holistic understanding of how software engineering work actually happens. Her evolution from DORA's focused pipeline metrics to SPACE's comprehensive framework demonstrates the industry's growing sophistication in recognizing that developer productivity is fundamentally about human experience, system design, and organizational culture rather than just code output counting.

The most significant insight from Nicole's work is how invisible much of valuable software engineering work remains to traditional measurement approaches. Senior engineers spend their time unblocking others, providing architectural guidance, and enabling team success in ways that never appear in commit logs or pull request counts. This reality demands measurement frameworks that capture satisfaction, collaboration, and system effectiveness alongside activity metrics, creating a more complete picture of how teams actually create value.

Looking toward the future, AI's integration into software development workflows represents both an opportunity and a challenge for productivity measurement. While traditional DORA metrics remain relevant for pipeline optimization, teams must now grapple with new dimensions like trust, verification, and workflow adaptation as AI fundamentally changes the nature of coding work. The key is maintaining focus on developer experience and team effectiveness while being open to dramatically different ways of working as technology continues evolving at unprecedented speed.

Practical Implications

• Implement constellation metrics combining at least three SPACE dimensions rather than relying on single productivity indicators like PR counts
• Measure onboarding time as a systems health indicator since fast new hire integration reflects overall tooling and documentation quality
• Focus platform team investment proposals on specific leverage points with quantified time waste data paired with concrete developer experience stories
• Build psychological safety through question-encouraging cultures that reward curiosity and continuous learning over defending existing approaches
• Experiment with AI coding tools individually to find personal workflows that preserve flow state while leveraging automation effectively
• Track developer satisfaction systematically through regular surveys asking about tool effectiveness and work experience quality
• Document and automate environment setup to minimize manual configuration requirements for new team members and internal transfers
• Create dummy pull request exercises for new hires to surface tooling problems and accelerate system familiarity
• Invest in developer experience improvements that reduce cognitive load and friction in daily work rather than just adding new features
• Align incentives to value platform work equally with feature delivery in performance reviews and recognition systems
• Conduct listening tours before proposing changes to understand current pain points and build support networks for improvement initiatives
• Acknowledge platform investment tipping points and commit to ongoing value assessment rather than treating developer tooling as unlimited resource sinks