Advertise on podcast: Engineering Enablement by DX

As AI agents become more capable, context quality is increasingly the limiting factor in their performance. In this episode of Engineering Enablement, DX researcher Brian Houck by his CAFE(S) is joined by co-authors Max Kanat-Alexander (Capital One), Eirini Kalliamvakou (DX), and Margaret-Anne Storey (University of Victoria) to discuss CAFE(S), a new framework for improving context quality. The panelists introduce the five dimensions of CAFE(S): Clarity, Actionability, Fidelity, Efficiency, and Security. They walk through common context failures, how poor context accrues technical and cognitive debt, what it costs in developer productivity and operating spend, how AI is changing the skills engineers need, and why context review could become as important as code review. Where to find Margaret-Anne Storey: • LinkedIn: https://www.linkedin.com/in/margaret-anne-storey  • X:  https://x.com/margaretstorey  • Website: https://margaretstorey.com Where to find Max Kanat-Alexander:  • LinkedIn: https://www.linkedin.com/in/mkanat • X: https://x.com/mkanat • Website: https://www.codesimplicity.com Where to find Eirini Kalliamvakou:  • LinkedIn: https://www.linkedin.com/in/eirini-kalliamvakou-1016865  • X: https://x.com/irina_kAl Where to find Brian Houck:  • LinkedIn: ⁠⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠⁠ In this episode, we cover: (00:00) Intro (01:18) Meet the CAFE(S) authors (03:13) Introducing CAFE(S): Why context quality matters (06:32) Common context failure modes (08:30) The five dimensions of high-quality context (12:33) When model failures are actually context failures (18:24) Why the panel prioritizes clarity in CAFE(S) (22:34) How context quality relates to technical, cognitive, and intent debt (25:35) How human interventions reveal context failures (28:11) How context quality relates to AI fluency (30:28) How AI is changing what makes a good engineer (33:48) The case for context review as a discipline  (38:44) Wrap-up Referenced: • CAFE(S): Your Agent Is Only as Good as Its Context • Nicole Forsgren • The Productivity-Experience Paradox - Annie Vella • Startup Studio • From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI • The AI-native developer • CodeScene

Meta researcher Moritz Beller joins Brian Houck to discuss how Meta measures developer productivity. Moritz explains diff authoring time and how Meta uses it to evaluate tools and guide engineering decisions. They explore how AI is changing engineers’ work and revealing gaps in traditional metrics. They also revisit Moritz’s “Mind the Gap” study and discuss the promise and risks of AI agents for testing. Where to find Moritz Beller: • LinkedIn: https://www.linkedin.com/in/inventitech  • X: https://x.com/Inventitech  • GitHub: https://github.com/Inventitech  • Website: https://inventitech.com Where to find Brian Houck:  • LinkedIn: ⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠ In this episode, we cover: (00:00) Intro (02:10) Moritz’s role at Meta (04:01) Measuring diff authoring time (08:20) Measuring A/B experiments (12:11) What diff authoring time reveals (14:45) Planning and leadership reporting (18:36) Why Meta measures teams, not individuals (22:33) Where developer time goes with AI (26:06) AI’s impact on junior and senior developers (27:18) Capturing invisible work with AI (29:32) The “Mind the Gap” study (33:40) Revisiting “Mind the Gap” in 2026 (38:32) AI agents for software testing (41:49) What remains hard to measure Referenced: • State of AI Impact in Engineering Q2 Report 2026 • GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis • From Technical Debt to Cognitive and Intent Debt  • Mind the Gap: On the Relationship Between Automatically Measured and Self-Reported Productivity • The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study • When, how, and why developers (do not) test in their IDEs | Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering • DORA, SPACE, and DevEx: Which framework should you use • A Tale of Two Cities: Software Developers Working from Home During the COVID-19 Pandemic

In this episode of Engineering Enablement, Robert Lucero, Chief Architect at Okta, joins host Brian Houck to discuss how agentic AI is changing identity and access management. They explore how organizations should define agent identities, why authentication and authorization remain foundational, and how sandboxing, fine-grained permissions, and just-in-time access can enable agents to operate autonomously without creating unacceptable risk. Robert shares what Okta has learned from driving AI adoption among security-minded engineers, what makes a repository ready for AI agents, and why strong testing, CI, and review processes matter even more as AI generates more code. They also discuss AI’s role in software validation, the need for human judgment, and whether AI will ultimately give the advantage to security teams or attackers. Where to find Robert Lucero: • LinkedIn: https://www.linkedin.com/in/rlucero Where to find Brian Houck:  • LinkedIn: ⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠ In this episode, we cover: (00:00) Intro (02:16) Meet Robert Lucero (02:56) Why identity becomes critical as AI agents access more systems (06:07) Governing automated access for non-deterministic agents (08:01) Defining and managing agent identities (10:32) Authentication and authorization for AI agents (13:33) Determining how much autonomy to give AI agents (19:44) Identity systems as the control plane for AI agents (22:40) Driving AI adoption among security-minded engineers (25:48) Why AI’s impact on engineering extends beyond coding (28:44) The three components of AI readiness for repositories (30:55) AI’s role in software testing and validation (34:26) Whether AI gives defenders or attackers the advantage (36:01) Where to find the best food in the country Referenced: • State of AI Impact in Engineering Q2 Report 2026 • Okta • The Hugging Face incident and the road ahead | OpenAI • GitHub Copilot · Your AI pair programmer • Claude Code by Anthropic | AI Coding Agent, Terminal, IDE • Eight Myths on Software Engineering and GenAI | Queue • Adriana Corona

Max Kanat-Alexander is an Executive Distinguished Engineer at Capital One, where he helps lead developer experience and AI-assisted software development. In this episode of Engineering Enablement, Max joins Brian Houck to discuss how AI is changing software development and why strong engineering fundamentals matter more than ever. They explore how engineering skills are evolving, why AI can amplify weaknesses in the development lifecycle, and how teams should rethink code review, quality, and testing. Max also shares how leaders can assess whether their organizations are ready for more advanced AI workflows and why developing the next generation of senior engineers remains one of the biggest open questions in software engineering. Where to find Max Kanat-Alexander: • LinkedIn: https://www.linkedin.com/in/mkanat  • X: https://x.com/mkanat  • Blog: https://www.codesimplicity.com  Where to find Brian Houck:  • LinkedIn: ⁠https://www.linkedin.com/in/brianhouck⁠ In this episode, we cover: (00:00) Intro (01:54) Max’s role at Capital One (02:52) Where to invest in engineering organizations   (06:36) The new entry-level engineering skills to pay attention to  (10:39) Why deepening your understanding still matters  (12:29) The bottlenecks around code review  (19:40) Why human code reviews still have value  (25:20) Why not all PRs need human review (26:11) The vicious cycle of AI-driven development (30:13) Using LLMs for refactoring  (33:58) AI readiness and why fixing engineering systems is so hard (38:42) Why research is needed on creating good senior engineers (41:00) Why AI will increase the need for engineers Referenced: • DX Core 4 Productivity Framework • Capital One  • Code Review Guidelines at Google  • Code Simplicity by Max Kanat-Alexander • How Microsoft sees engineering bottlenecks changing with AI

AI adoption among software developers is approaching 100%, AI-authored code now makes up more than half of merged code, and developers report saving more time with AI every quarter. But those gains aren’t translating evenly into better outcomes. In this episode of Engineering Enablement, host Brian Houck, Distinguished Scientist at DX, sits down with Justin Reock, Deputy CTO at DX, to unpack findings from DX’s latest AI Impact Report. They explore where AI is improving engineering velocity and developer experience, where concerns are emerging around PR size, change confidence, and failure rates, and why rising AI spend has yet to produce a comparable increase in innovation. They also discuss how AI is changing the meaning of code maintainability and where developers’ AI-driven time savings may actually be going. Where to find Justin Reock: • LinkedIn: https://www.linkedin.com/in/justinreock Where to find Brian Houck:  • LinkedIn: https://www.linkedin.com/in/brianhouck In this episode, we cover: (00:00) Intro (01:45) How the current AI impact report is tied to Core 4  (03:24) The state of AI adoption (05:12) How much time AI is saving developers and percentage of AI-authored code (07:47) AI’s impact on PR throughput and deployment frequency (11:09) How EMs are shipping more code (13:02) Why larger PRs may be problematic (18:21) The growing gap between code maintainability and change confidence (21:48) How perceived code quality varies by organization size (23:49) The growing volatility in change failure rates (28:07) What the Developer Experience Index reveals (32:20) Cost, dev ramp-up, and innovation ratio  (35:38) Where AI time savings are getting lost (37:11) Questions and wrap-up Referenced: • DX Core 4 Productivity Framework • AI Impact report • The AI-native developer - by Brian Houck • GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis • Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools | NBER  • The Productivity-Experience Paradox - Annie Vella • EngThrive: Make It Fast and Easy to Do Great Work • The AI efficiency plateau - by Brian Houck • Tradable Quality Hypothesis

Tim Bozarth is a Corporate Vice President in Microsoft CoreAI, where he leads engineering for next-generation developer experiences and Microsoft's Engineering Thrive initiative. Throughout his career at Microsoft, Google, Netflix, and Box, he has focused on developer productivity, engineering systems, and organizational effectiveness. In this episode of Engineering Enablement, Tim joins host Brian Houck to discuss Engineering Thrive, Microsoft's framework for measuring and improving engineering productivity. They explore why AI makes outcome-based metrics more important than ever, where new bottlenecks are emerging in the software development lifecycle, and why verification and confidence may become more valuable than code generation itself. Tim also shares why the purpose of engineering remains the same despite rising levels of abstraction, the skills that remain durable in an AI-driven world, and why engineering leaders should focus on outcomes rather than activity metrics. Where to find Tim Bozarth: • LinkedIn: linkedin.com/in/tbozarth Where to find Brian Houck:  • LinkedIn: ⁠https://www.linkedin.com/in/brianhouck⁠ In this episode, we cover: (00:00) Intro (02:13) What Engineering Thrive is and the problem it solves (04:46) Why Engineering Thrive isn't specific to Microsoft (09:10) The impact of Engineering Thrive at Microsoft (14:31) Why AI makes outcome-based productivity metrics more important (18:22) Where AI is creating new bottlenecks in the SDLC (24:37) Why more abstraction doesn't change the purpose of engineering (27:25) The durable skills of good engineers  (33:03) The changing economics of software development (36:56) Advice for leaders: measure outcomes, not activity Referenced: • DX Core 4 Productivity Framework • EngThrive: Make It Fast and Easy to Do Great Work: Building a durable model for outcome-oriented engineering measurement • Quote by Eliyahu M. Goldratt: “Tell me how you measure me and...” • Jevons paradox - Wikipedia

AI coding agents are changing how software gets built, but they're also forcing organizations to rethink how engineering effectiveness is measured. Traditional developer experience metrics were designed for humans, not AI agents, so how should engineering leaders adapt? In this webinar, Justin Reock, Deputy CTO at DX, is joined by Brian Houck, Distinguished Scientist at DX and co-author of the SPACE framework, to explore the emerging field of agent experience and how it builds on developer experience rather than replacing it. They discuss how organizations can prepare for AI-assisted software development, how the DX Core 4 applies in the age of AI, why metrics like token usage and PR throughput don't tell the whole story, and the growing importance of documentation. They also examine the impact AI-driven pressure is having on burnout and cognitive overload. Throughout the conversation, they share practical guidance for building engineering organizations where both developers and AI agents can do their best work. Where to find Brian Houck:  • LinkedIn: https://www.linkedin.com/in/brianhouck Where to find Justin Reock: • LinkedIn: ⁠https://www.linkedin.com/in/justinreock⁠ In this episode, we cover: (00:00) Intro (01:26) Justin’s new role at DX (03:53) What agent experience is and why engineering leaders should care (08:55) How to improve agent experience at the platform level (11:25) How agent experience and developer experience influence each other (14:41) Preparing engineering teams for agentic work (21:38) Why the DX Core 4 still matters in the age of AI (27:23) What PR throughput actually measures  (32:47) The limits of token metrics (37:10) What the data shows about documentation and developer experience (39:32) Improving documentation for AI agents (40:43) AI-washing, burnout, and cognitive overload (45:35) Brian's upcoming research on agent experience Referenced: • DX Core 4 Productivity Framework • The SPACE of AI | Queue • Sara Chizari on LinkedIn • The Middle Loop - Annie Vella • gastownhall/gastown: Gas Town - multi-agent workspace manager · GitHub • DORA, SPACE, and DevEx: Which framework should you use? • AI Impact Report: Q1 2026 • AI-authored code has nearly doubled, but so has PR size  • EngThrive: Make It Fast and Easy to Do Great Work: Building a durable model for outcome-oriented engineering measurement • Google's Project Aristotle - Psychological Safety  • Zapier

Sejal Amin is the Chief Technology Officer at Priceline, where she leads product engineering, infrastructure, data, and technology operations. Pedro Gutierrez is Senior Director of Software Engineering, where he has helped drive developer experience initiatives and the adoption of Priceline's product operating model. In this episode of the Engineering Enablement podcast, Justin Reock talks with Sejal and Pedro about Priceline's journey from a project-based organization to a product operating model and the role developer experience played in making that transformation successful. They discuss how DX metrics and developer feedback helped identify organizational bottlenecks, guide structural changes, empower engineering managers, and build trust across teams. They also explore the importance of clear communication, creating a dedicated developer experience team, and how their operating model has helped prepare Priceline for AI-driven software development. Where to find Sejal Amin:   • LinkedIn: https://www.linkedin.com/in/sejal-amin Where to find Pedro Gutierrez:  • LinkedIn: https://www.linkedin.com/in/pedro-gutierrez-b6605422 Where to find Justin Reock: • LinkedIn: https://www.linkedin.com/in/justinreock In this episode, we cover: (00:00) Intro (01:07) Meet Sejal Amin and Pedro Gutierrez (01:47) How Priceline's developer experience journey began (04:55) Lessons from Priceline's first developer experience surveys (06:55) How DX improved Priceline's developer experience surveys (09:47) Identifying the causes of organizational slowness (12:33) How the product operating model changed the way Priceline works (14:10) Priceline's phased rollout with DX (18:14) How DX insights drove organizational changes (19:33) Why Priceline improved developer experience before org change was complete (22:18) How clear communication builds trust (24:25) Early results from Priceline's Core Four (25:38) Creating a culture of continuous feedback to build trust (27:40) What has changed in the engineering manager role (30:10) Resources for learning about the product operating model (32:40) What Pedro learned from implementing DX (34:51) The developer experience team (35:59) How AI tools have impacted Priceline’s teams  (37:20) How the product operating model supports AI-driven development (39:13) Final advice for engineering leaders Referenced: • Measuring developer productivity with the DX Core 4 • Transformed: Moving to the Product Operating Model (Silicon Valley Product Group)  • Team Topologies • Flow Framework • Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework

In this closing panel from DX Annual, Rafe Colburn, Chief Product and Technology Officer at Etsy; Jesse Adametz, Senior Director of Engineering, Platform Engineering at Twilio; Eirini Kalliamvakou, Research Advisor at GitHub; Collin Green, Senior Staff UX Researcher at Google; and Brian Houck, Senior Principal Applied Scientist at Microsoft debate some of the biggest questions surrounding AI and engineering productivity. They discuss whether AI will reduce the need for engineers, how AI is affecting technical debt, the future role of software engineers in an agentic world, and whether organizations should mandate AI adoption. They also explore how bottlenecks are shifting across the software development lifecycle, the challenges facing junior engineers, and why learning, culture, and change management may ultimately matter more than the tools themselves. Where to find Rafe Colburn: • LinkedIn: https://www.linkedin.com/in/rafeco • Blog: https://rafe.codes Where to find Eirini Kalliamvakou:  • LinkedIn: https://www.linkedin.com/in/eirini-kalliamvakou-1016865 • X: https://x.com/irina_kAl Where to find Brian Houck:  • LinkedIn: https://www.linkedin.com/in/brianhouck Where to find Jesse Adametz:  • LinkedIn: https://www.linkedin.com/in/jesseadametz  • X: https://x.com/jesseadametz  • Website: https://www.jesseadametz.com  Where to find Collin Green:  • LinkedIn: https://www.linkedin.com/in/collin-green-97720378 In this episode, we cover: (00:00) Intro (01:16) Why an AI-first SDLC doesn’t mean fewer engineers  (03:09) The debate over AI and technical debt (07:40) AI-generated code and the future role of engineers (14:16) Why mandating AI use doesn't necessarily lead to better outcomes (20:43) Predictions for the future of junior engineers  (23:22) Where the bottlenecks are in the SDLC now (28:25) How risk influences AI use  (32:38) Why the human side is the biggest AI adoption challenge Referenced: • Etsy • GitHub • Microsoft • Twilio • Google  • Stewart Reichling • What is the SPACE framework and when should you use it?

In this session from DX Annual, Uma Namasivayam, Senior Director of Engineering Productivity at Dropbox, shares how the company's developer productivity efforts evolved from improving developer experience to preparing for the agentic era. He explains how Dropbox approached AI adoption across its engineering organization, the impact it had on developer productivity, and why faster code generation is creating new bottlenecks in areas such as code review, validation, and CI/CD. He also discusses Dropbox's efforts to rethink engineering systems, measurement, and workflows, including the development of agentic tooling and new metrics designed to move beyond PR throughput and toward product velocity. Where to find Uma Namasivayam: • LinkedIn: https://www.linkedin.com/in/unamasivay In this episode, we cover: (00:00) Intro (00:57) The beginning of Dropbox’s DX journey (02:34) AI adoption at Dropbox: what made it work   (04:46) The results of Dropbox's AI adoption efforts (05:39) What the results mean for the business  (06:55) The phases of AI adoption and where they are now (08:00) The new bottlenecks (09:16) Three challenges Dropbox faces moving into agentic engineering (10:05) How Dropbox is redesigning the SDLC for agentic engineering (15:46) The new metrics that matter  (19:16) Final takeaways Referenced: • Dropbox  • Developer Experience Index (DXI) | DX  • DX Core 4 Productivity Framework • Cursor • Claude Code | Anthropic's agentic coding system • JetBrains  • Visual Studio Code • Jira | Project Management for the AI Era | Atlassian • GitHub

Indeed increased AI coding tool adoption from roughly 25% to 97% across its engineering organization, but getting engineers to use the tools was only part of the challenge. In this session from DX Annual, Michael Redding, Principal Product Manager, and Jeff Davis, VP of Core Infrastructure at Indeed, explain how the company used structured training, leadership support, and ongoing community engagement to help more than 2,000 engineers build practical AI skills. They share why an early train-the-trainer model fell short, how they redesigned their approach around hands-on learning, and what they learned about balancing adoption, measurement, and psychological safety. They also discuss the impact of the program on coding time, the role of continuous enablement after formal training ended, and how Indeed is preparing for the next phase of AI adoption, including agentic workflows and AI-powered coaching. Where to find Jeff Davis:  • LinkedIn: https://www.linkedin.com/in/utjeffd  Where to find Michael Redding: • LinkedIn: https://www.linkedin.com/in/reddingsetgo In this episode, we cover: (00:00) Intro (01:05) Indeed's DX survey from January 2025 (02:30) The two-part strategy to double engineering productivity (04:21) How Indeed increased AI adoption from 25% to 97% (15:40) Results from Indeed's AI training program (18:33) How Indeed sustains AI adoption and learning (23:06) What's next for AI enablement at Indeed (24:41) Q&A: How coding time was calculated (25:25) Q&A: How Indeed uses AI playbooks (26:40) Q&A: Balancing asynchronous and live AI training (28:22) Q&A: Psychological safety during AI adoption (31:44) Q&A: Why AI adoption spikes after the holidays (33:20) Q&A: The metrics Indeed tracked  (35:22) Q&A: Where the time savings are going  (36:54) Q&A: Reaching engineers who skipped the training (38:08) Closing thoughts Referenced: • Indeed • Claude Code | Anthropic's agentic coding system • Cursor • Windsurf • Amp Code • The Complete Guide to Building Skills for Claude | Anthropic • Measuring developer productivity with the DX Core 4

In this session from DX Annual, Rebecca Fitzhugh, Lead Principal Engineer at Atlassian, moderates a panel featuring Nidhi Allipuram, Vice President, Enterprise Developer Experience and Platform at Nationwide, Jai Schniepp, Senior Director, DevX Product Management at Comcast, Brent Foster, Vice President and Head of Architecture and Strategy at TD Bank, and Praveena Patchipulusu, Vice President of Engineering at HPE. Together, they discuss how large enterprises are approaching AI adoption, what it takes to build an AI-first software development lifecycle, and how engineering leaders are balancing speed, security, governance, and developer experience. They also share their perspectives on the changing role of engineers, human accountability, and how organizations can prepare for the future of software engineering. Where to find Rebecca Fitzhugh:  • LinkedIn: https://www.linkedin.com/in/rmfitzhugh  • X: https://x.com/RebeccaFitzhugh  Where to find Jai Schniepp: • LinkedIn: https://www.linkedin.com/in/jessicaschniepp Where to find Nidhi Allipuram:  • LinkedIn: https://www.linkedin.com/in/nidhi-allipuram Where to find Brent Foster:  • LinkedIn: https://www.linkedin.com/in/engineeringthefuture • Website: https://brentfoster.me Where to find Praveena Patchipulusu:  • LinkedIn: https://www.linkedin.com/in/praveena-patchipulusu-158741 In this episode, we cover: (00:00) Intro (02:28) The AI journey across TD Bank, Comcast, and HPE (05:59) Inside Nationwide's AI-assisted development lifecycle (10:04) Reimagining the software development lifecycle with AI (11:32) Security, governance, and human accountability (15:27) Embedding security and guardrails into AI workflows (17:55) How AI is changing the role of an engineer (21:52) What developer experience looks like in the AI era (26:55) What software engineering may look like in 2030 (32:47) How to prepare for the AI-driven future Referenced: • Atlassian • TD Bank • Comcast Corporation • Hewlett Packard Enterprise (HPE) • Nationwide  • GitHub Spec Kit • Abi Noda

In this session from DX Annual, Christopher Sanson, Product Lead, AI Developer Experience, and Madison Capps, Engineering Manager, Infrastructure at Airbnb, challenge some of the most common assumptions about AI. Is AI primarily about replacing humans? Do organizations need mandates to drive adoption? And are the productivity gains really as small as some studies suggest? Using examples from Airbnb's own AI journey, they share how the company achieved widespread adoption of agentic AI through AirChat, community enablement, and internal tooling rather than top-down mandates. They also discuss the impact AI is having on developer productivity, how non-developers are increasingly using coding tools, and how teams are rethinking product development in an AI-first world. Finally, Madison takes a deeper look at the infrastructure powering Airbnb’s AI strategy, including AirChat CLI, the AirChat SDK, and AirChat Remote, along with the company’s vision for asynchronous agent workflows and the next generation of AI-powered development. Where to find Christopher Sanson: • LinkedIn: https://www.linkedin.com/in/christophersanson  Where to find Madison Capps: • LinkedIn: https://www.linkedin.com/in/madison-capps-66950625 In this episode, we cover: (00:00) Intro (01:37) Myth #1: AI is about replacing humans (03:22) Myth #2: You need mandates to drive AI adoption (05:21) AirChat, agentic AI, and Airbnb's adoption strategy (08:07) Myth #3: AI has little impact on productivity (09:33) Airbnb's increase in coding time and PR throughput (14:20) Myth #4: AI coding tools are just for coders (15:39) How non-developers are using coding tools (17:24) Rethinking product development in an AI-first world (20:30) Myth #5: Vibe coding isn’t coding (22:16) Unsolved problems in agentic AI tooling and how Airbnb is addressing them (26:30) Airbnb’s overall AI philosophy in practice (29:15) Using agentic AI to accelerate code migrations (30:18) AirChat SDK: How Airbnb enables teams to build AI-powered applications (33:17) AirChat Remote and asynchronous agent workflows (36:07) Predictions for what’s next Referenced: • ⁠Airbnb • Steve Jobs’s Bicycles for the Mind  • Jennifer St Pierre  • Justin Reock • AI-generated merged code holds steady at ~30% • Andrej Karpathy's post on X

As AI becomes embedded in software development, many of the metrics that engineering organizations have relied on for years are starting to break down. In this session from DX Annual, Uber's Ty Smith and Abhishek Tibrewal share how their approach to measuring AI's impact on developer productivity has evolved over time. They walk through the different phases of their measurement journey, from adoption and engagement to measuring impact, ROI, and agentic value, explaining what they chose to measure at each stage, what worked, what failed, and how their thinking changed along the way. They also discuss the role of qualitative feedback before telemetry existed, the challenge of identifying meaningful engagement signals, why "developer years saved" failed as an ROI metric, and how AI agents forced them to rethink traditional productivity measurements. Finally, they introduce Uber's emerging framework built around feature velocity and explore the unanswered questions that remain as software development becomes increasingly agent-driven. Where to find Abhishek Tibrewal  • LinkedIn: https://www.linkedin.com/in/aabhishektibrewal Where to find Ty Smith:  • LinkedIn: https://www.linkedin.com/in/tyvsmith In this episode, we cover: (00:00) Intro (01:30) Steve Yegge’s 8 stages of AI-assisted development  (03:22) Uber’s shift to a generative AI-powered company  (04:20) Uber’s pre-AI productivity metrics  (06:55) Important questions from stakeholders that previous metrics didn’t answer  (08:25) How Uber measures AI before telemetry exists (11:11) Metrics used to measure adoption (12:49) Measuring engagement (14:30) Measuring impact (16:32) The challenge of measuring AI ROI (19:32) Rethinking adoption, engagement, and impact for agentic AI (26:01) The new north star: Feature velocity  (28:41) PR classification + feature velocity: the questions it can answer  (33:01) What comes next and what’s still unanswered  (34:30) Lessons learned and what they'd do differently (37:11) Q&A #1: How Uber defines a feature  (38:50) Q&A #2: Measuring success and AI ROI Referenced: • Welcome to Gas Town • Dara Khosrowshahi (Uber CEO)

Kelly Anne Pipe is Head of Developer Experience at Vanguard, and Nicole Scribner is a Director in the firm's Chief Technology Office focused on engineering enablement and advancement. In this session from DX Annual, Kelly Anne and Nicole share how Vanguard is expanding its AI strategy beyond software engineering to the entire product development lifecycle. While the company initially focused on tools like GitHub Copilot for engineers, they found that faster coding alone did not significantly improve delivery speed. Product managers, designers, QA teams, and organizational processes were still operating at a different pace. To address this challenge, Vanguard developed a product team maturity model built around three stages: Augmented, Accelerated, and Autonomized. The framework spans six dimensions, from AI-powered delivery and AI-ready codebases to team autonomy, operations, and responsible AI. Kelly Anne and Nicole explain how Vanguard is applying the model across more than 800 product teams, the behaviors they believe will enable faster delivery, and the lessons they have learned about measurement, organizational change, dependencies, and scaling AI across the product development lifecycle. In this episode, we cover: (00:00) Intro (02:16) The state of AI one year ago at Vanguard (02:54) The engineering bubble (05:05) Building an AI maturity model for 800 product teams (08:24) Dimension 1: AI-powered product delivery (10:00) Dimension 2: AI-ready codebase (12:20) Dimension 3: Autonomous agent utilization  (13:00) Dimension 4: AI-augmented operations (14:00) Dimension 5: Team autonomy and enablement (16:11) Dimension 6: Responsible AI (18:15) The people problem: role evolution  (20:00) The measurement problem  (22:55) Lessons learned from rolling out the maturity model  (26:46) What’s ahead  (30:10) Q&A #1: Getting your codebase ready for AI (32:22) Q&A #2: Audit trails and responsible AI (34:16) Q&A #3: Vanguard's maturity model progress (36:15) Q&A #4: Measuring cycle time across 800 teams Referenced: • Vanguard • Jennifer St Pierre - Dell Technologies | LinkedIn • Mercari