1505372978
MLOps.community

Advertise on podcast: MLOps.community

Rating
★★★★★
4.6
from
25 reviews
Categories
Country
United States
This podcast has
566 episodes
Language
English
Publisher
Demetrios
Explicit
No
Date created
2020/03/30
Latest episode
2026/11/16
Average duration
38 min.
Release period
4 days

Description

Relaxed Conversations around getting AI into production, whatever shape that may come in (agentic, traditional ML, LLMs, Vibes, etc)

Unlock MLOps.community podcast Email contact info,
Listeners & Audience details

Email contact information

Direct podcast contact details

Listeners

Audience numbers & engagement insights

Audience details

Podcast Insights

Social media

Check MLOps.community social media presence


Podcast episodes

Check latest episodes from MLOps.community podcast


Multi-Cloud LLM Routing and Context-Aware Coding Agents
2026/11/16
In this episode, we sit down with Matt Carey and Scott Crooks, engineers at Cloudflare and Miro, to look at how two very different companies are building their own AI software factories, and what happens to engineering when the code mostly writes and reviews itself. We start with the AI gateway: why a model call is no longer simple when the same model is served across multiple regions, multiple clouds, and direct from the provider, and how Miro pools rate limits and data residency rules (like EU-only models for German customers) into a single routing layer that also powers Miro’s Sidekicks. Matt walks through Cloudflare’s agent stack, from the AI gateway and Agents SDK to Flue, a JavaScript agent framework from the makers of Astro, and Cloudflare OS, an open source agent office suite he even runs at home for grocery shopping. The heart of the conversation is code review. Matt describes what he calls the best AI code reviewer he’s ever used, built on Flue and powered by an internal engineering codex written only for agents. Scott breaks down Miro’s three-layer setup, including Easel, a bot that reads production logs overnight and opens fix PRs before anyone wakes up, and explains why only about half of those PRs get merged, and how that signal is fed back to teach the system what people care about. Along the way, we cover agent experience as a new discipline, how open source maintainers are handling a flood of cheap AI pull requests, Astro reaching zero open issues, when companies should open source internal tools and how to retire them gracefully, plus a few hot takes on chocolate and nuts. We close on the rise of the “scholarly engineer,” and why understanding ideas may matter more than reading every line of code. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmMatt Carey: https://www.linkedin.com/in/mattzcareyScott Crooks: https://www.linkedin.com/in/jshcrooks Timestamps:[00:00] Cold open[01:03] Meet Matt and Scott[01:20] What an AI gateway really is[02:28] Multi-region, multi-cloud model routing[05:00] Cloudflare’s agent stack and Cloudflare OS[06:43] The Flue framework from the Astro team[07:32] The best AI code reviewer[09:29] An engineering codex written for agents[11:48] Agent experience and evals[12:44] Miro’s Easel bot and three layers of review[15:55] Software factories and Astro hitting zero issues[18:58] Companies open sourcing internal tools[22:59] Hot takes[25:03] The scholarly engineer[28:50] Wrap up
Why Stripe Cut 500,000 Internal Docs Down to 5,000?
2026/11/13
What happens when AI agents start relying on the same messy docs your team already struggles to search? Demetrios hops into a Waymo in San Francisco with Dave Nunez, founder and CEO of Falconer and co-author of Documentation for Developers. They talk about why developers and agents need different things from docs, why agents sometimes skip up-to-date instructions, and how a stale internal document can contribute to a production incident. Dave shares how Stripe cut roughly 500,000 internal docs down to 5,000 worth maintaining. The conversation also covers how to capture the reasoning behind engineering decisions, when a diagram beats a wall of text, and who owns the fallout when an agent acts on bad information. Timestamps: [00:00] Hopping into a Waymo in San Francisco [00:24] Meet Dave, founder of Falconer [01:15] Why internal docs matter for AI agents [03:24] What humans and agents each need from docs [05:46] Testing how well agents use developer tools [07:01] Why agents skip the latest docs [09:42] Decision records and the reasoning behind the code [14:04] Cleaning up stale docs and broken search [15:29] How Stripe went from 500,000 docs to 5,000 [17:02] When a diagram beats a 10-page doc [18:53] How conflicting docs can lead to an incident [20:42] Who owns reliability when an agent makes a mistake? [24:01] Treating documentation as part of the product [26:38] Giving readers an escape hatch [27:27] The cost of AI-generated fluff [29:36] Wrap-up and post-show chat
My AI Agent Bought Me A Car
2026/11/09
Sam Herring’s first real test of Hermes Agent was a shopping tool that matched clothes to photos of ancient pottery. A few months later, the same agent was checking government auction sites every day until it found him a retired three-wheel parking ticket car, which he now drives around San Francisco. In this episode, recorded at AGNTCon + MCPCon North America in San Jose, Sam, who works on the RL post-training team at Nous Research, goes from those everyday wins to the question everyone in AI is asking: can agents improve themselves? He explains how Hermes grew from an internal productivity tool into one of the most talked-about open-source agents, and why letting it write its own skills was the moment it clicked for users. He also describes an early version that could design its own RL environment, train on Thinking Machines’ Tinker, and hot-swap the new weights into a model running locally. Sam then breaks down Trace to Environment, an idea he credits to Xiaomi’s MiMo-V2.6-Pro work, and the loop Nous is building around it: find the exact point where an agent fails, cluster those failures by type, synthesize environments that recreate them, and train on them again and again. The second half moves from training to inference. We get into whether reasoning models overthink, why leaning on pretraining knowledge can be a model’s Achilles heel, and why choosing a reasoning effort level may be a temporary quirk. Sam also makes his case for activation steering as an underexplored way to get more out of small models, explains where steering meets interpretability, and describes what true continual learning could look like. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmSam Herring: https://www.linkedin.com/in/sam-herring-67225819a Timestamps:[0:00] Cold open[1:21] Meet Sam from Nous Research[2:39] How Hermes Agent started[3:38] From pottery to a parking ticket car[7:40] Benchmarking life admin tasks[8:58] Agents that write their own skills[10:10] RL at Nous a year ago[12:24] Hot swapping self-trained weights[15:00] Xiaomi and Trace to Environment[17:22] Turning agent failures into training[20:27] Where the RSI conversation is heading[21:34] Token efficiency and overthinking[26:30] Steering models at inference time[32:35] Toward true continual learning[34:54] Wrap up
Cut One AI Cost and Two More Grow Back
2026/11/05
What does AI really cost once thousands of people inside a company start using it? Ioannis Zempekakis, Director of AI at Prosus, has been running agents in production since 2024. We put that question to him on a tandem bike ride from a favorite Amsterdam cafe to Vondelpark. Ioannis explains the decision to stop relying purely on cloud APIs, and why that approach breaks down at the scale of a large company. He covers why agent harnesses and sandboxes are as greedy as software gets, and why agents scheduled to run every five minutes repeat a lesson the data world already learned about freshness. He also explains how the habit of throwing away a codebase and prompting again quietly adds up. His bigger point is that even if model progress froze tomorrow, there would still be about two years of value left to squeeze out. The cheapest wins come from controlling context and running a basic model router, while ten new features a month keep sprouting like Hydra heads. From there we get into when fine-tuning small models pays off for repetitive work like data analytics, and the eval suites that come as a byproduct of building in-house. Ioannis explains how owning the stack let Prosus grow a GPT-3 Slack bot into a full agent harness. He also explains why committing to a single vendor or open source project is a bigger bet than it looks, and how to balance top-down priorities with bottom-up building across the company. We close on hiring. Ioannis looks for system design over syntax, a nose for the over-engineered code AI models love to produce, and enough agent intuition to know when to build something yourself and when to let the agent decide. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm Ioannis Zempekakis: https://www.linkedin.com/in/zempekakis Timestamps: [0:00] AI total cost of ownership on a bike [0:48] The million dollar decision [4:26] When everyone can build with AI [6:51] Which use cases are worth the tokens [10:46] Benchmarking every model on every task [12:18] Total cost of ownership and AI ROI [15:12] Context control and model routing [16:39] Training and fine-tuning your own models [19:41] Is the juice worth the squeeze [25:26] Optionality and vendor lock-in [27:30] From Slack bot to agent harness [30:12] Top-down vs bottom-up AI adoption [32:19] What AI engineer even means [35:13] What knowing AI really means [37:47] The hardest part of hiring
How Stanford Teaches AI Coding Now
2026/11/02
What does it take to build AI agents that deliver useful, reliable work? Stanford lecturer and Monaco Head of AI Mihail Eric joins Demetrios for a conversation inside a Waymo about how AI is changing software development - and what it takes to put agents into production. Mihail explains why he rebuilt most of his Modern Software Developer course, how he teaches students to avoid shipping AI slop, and why research, planning, and verification still matter as coding agents become more capable. They discuss Claude Code, agent loops, context engineering, memory and compaction, and how to design MCP tools around what an agent needs to accomplish. Then they go inside Monaco’s approach to sales agents, from turning meetings and emails into useful insights to building a custom agent harness. Along the way: lessons from Boris and Lee Robinson’s guest lectures, why company traces are becoming valuable assets, and Monaco’s unusual approach to brand building. What’s the biggest bottleneck in your agent workflows: context, tools, or verification? Let us know in the comments. Timestamps: [00:00] Meet Mihail Eric [00:16] Teaching AI Coding at Stanford [02:42] Avoiding AI Slop [04:48] Research, Plan, Build, Test [06:19] Verifying Agent Work [09:16] Lessons from Claude Code’s Creator [12:40] Proactive AI Assistants [16:13] Agent Memory & Context [17:41] Rethinking Git for Agents [19:32] AI Startup Pitches [22:51] Why Traces Matter [24:09] Better MCP Tools [26:01] Monaco’s Sales Agents [29:40] Turning Conversations into Insights [33:33] AI Meets Human Expertise [35:03] Poker & Brand Building [36:46] Inside Monaco’s Agent Stack [38:44] Designing Tools Around Intent [40:12] Building an AI Advantage [42:53] Wrapping Up
From Slop Code to Software Factories
2026/10/29
Two people, one tandem bike, and a lot of near misses along Amsterdam’s canals. Dexter Horthy, CEO and co-founder of HumanLayer and the person who coined “context engineering,” climbs on the back to talk about a problem every AI-heavy engineering team eventually hits: code that runs today that nobody understands three months later. The ride starts with an old joke from the Claude 3.7 days: ship the slop, it’ll be GPT 7’s problem. That joke is aging badly now that GPT 6 is out and still can’t design clean abstractions. From there it becomes a tour of how agents are actually used right now. Computer use checks in for flights and books government appointments. Two hours of raw footage became a rough but promising product demo through Opus 4.8 and the Palmier MCP while Dex was out for a run. And the best MCP tools turn out to be the ones that mirror what Claude Code and Codex were trained to call. Off the bike, over pour-over coffee, the questions get bigger. Are browser agents a bandaid until everything goes headless? What happens to ad-funded websites and upsells when an agent does the shopping? Why does chat still feel like such a low-bandwidth way to work with models? And why is every principal engineer quietly building their own software factory, then getting buried in the sync, streaming and approval plumbing that HumanLayer wants to turn into building blocks, the way GitHub Actions did for CI? Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm Dexter Horthy: https://www.linkedin.com/in/dexterihorthy Timestamps [00:00] Intro [01:38] The GPT 7 will fix it joke [03:03] GPT 6 still writes spaghetti [06:21] If it runs, does it matter [07:28] Computer use that fills out your forms [11:57] Editing a demo with Palmier MCP [13:16] Why most MCP servers are bad [14:32] Mirror the tools models were trained on [16:57] Hill climbing a demo while out running [23:49] Are browser agents a band-aid [32:23] Why chat needs richer interfaces [40:30] Every engineer is building a software factory [45:13] The GitHub Actions moment for agents
Tim O’Reilly
2026/10/26
A large language model knows everything and nothing at the same time. Tim O’Reilly, founder and CEO of O’Reilly Media, joins us ahead of his keynote at AGNTCon + MCPCon North America to talk about what that paradox means for the people building with these models. We start with what Tim is bringing to the stage in San Jose, then move to the craft of steering a model. The right words work almost like spells that open up new regions of latent space. Something as small as telling a model to use a computer can unlock a new tier of capability, where agents go off and do real work. From there, the conversation turns philosophical. Tim makes the case for mechanism design: shaping the rules of a system so the outcomes we want become the natural ones. We connect it to a book, The Score, and the question of which game we are really playing. In Risk, everyone fights over the board. In charades, everyone plays together. We close on Tim’s view that AI should be an ecosystem rather than a product that sucks the air out of the room, so that everyone can build on top of it and flourish. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm Tim O’Reilly: https://www.linkedin.com/in/timo3 Timestamps: [00:00] Cold open [01:02] Grappling with the bitter lesson [02:22] The value of human expertise [04:54] Why AI needs us [05:37] Culture is what remains [06:23] The toil audit story [08:26] AI knows everything and nothing [08:57] Magic words and runaway agents [11:18] Brute force vs human context [12:32] Turning scar tissue into infrastructure [14:12] Google and the first bitter lesson [16:25] Silicon Valley's sad turn [17:43] Artificial volition [19:08] Casting spells in latent space [20:46] The peaceful atom lesson [22:15] Sharing value with creators [24:54] No jobs means no customers [25:43] Mechanism design explained [26:35] Unix and the architecture of participation [28:45] The missing mechanisms of the AI economy [29:36] What game are you playing [30:43] The game of human flourishing [32:04] Big labs vs open source AI [33:47] Wrap up
AI Agents Need a Babysitter
2026/10/23
AI can write your code, choose its own tools, and book your next trip. But who checks that it actually did the job? In this episode of Debug Mode, we explore what it takes to give AI agents more responsibility—and how to keep track of what they’re doing. We discuss intent-based tool selection, MCP and WebMCP, and why a convincing demo doesn’t always translate into reliable automation. We also get into the personal AI assistant everyone seems to be building: one that manages travel, handles household tasks, and hopefully remembers to order coffee before you run out. Then it’s back to code: QA agents, automated verification, tests that capture business intent, and the growing challenge of reviewing AI-generated contributions. Timestamps:[00:00] Who gave AI the keys?[00:43] A tattoo years in the making[02:40] Tell the agent what you want—not which tool[06:19] MCP enters the agent harness[07:23] Great demo. What do the benchmarks say?[09:55] AI builders rediscover machine learning[11:50] Can WebMCP make agents more reliable?[16:04] Unlocking tools like a video game skill tree[18:19] Let an agent try to break your website[19:50] Are humans still in the driver’s seat?[20:38] Build for the outcome, not the API[24:32] Why can’t AI keep my coffee stocked?[27:38] A house manager for the rest of us[29:23] A travel agent that actually handles things[30:49] When your AI chat becomes your workspace[34:12] Does superintelligence need a rebrand?[35:06] Who checks the code AI writes?[36:31] Test why the code exists—not just if it works[41:08] Give your coding agent a QA partner[43:18] The checks failed. Why submit it anyway?[44:33] Open source meets the AI code flood
Token Maxxing Is Over and Here Is What Comes Next
2026/10/22
What happens when you give 200 engineers who have barely touched AI a coding assistant and a six-month deadline to rebuild their platform? Pepijn Senders, a principal engineer who previously led AI enablement at HelloFresh, has done exactly that, and he chose Sonnet over Opus on purpose. His reasoning is what he calls "Sonnet maxxing." A model that reasons less and stops sooner keeps developers involved, and that matters a lot when the codebase is banking software full of hard business rules. The result was 800,000 lines of code, about 60% of them written by Sonnet, shipped by a team that never stopped thinking for itself. Pep also explains how to justify the spend once the token-maxxing era ends and the CFO starts asking questions. He covers a scatter chart that plots token spend against output, the uncomfortable question of whether a 1% better home screen was worth $20K in tokens, why he measures gains in headcount rather than 2x or 3x multipliers, and where he fell short when trying to prove ROI at HelloFresh. We also get into a codebase test you can run yourself, why AI recommendations quietly narrow your choices, and multiplayer refinement sessions where the whole team works through the agent's questions together. Pep shares an experiment where designers and engineers got the same one-shot prompt, and it showed how words like "scalable" and "maintainable" steer a model. We finish with how his team built a meta harness on top of Claude Code, after finding that 85% of their code followed the same path: refinement, specification, implementation, verification. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm Pepijn Senders: https://www.linkedin.com/in/pepijn-senders Timestamps: [0:00] Can you still read your codebase [1:15] Meet Pep from HelloFresh [2:35] What is Sonnet Maxing [3:41] Rebuilding a platform in six months [5:02] Why Sonnet made adoption easier than Copilot [6:37] Proving the ROI of AI coding [8:50] Fighting for AI ROI at HelloFresh [11:02] Token spend vs output [13:24] Tinder pickup lines on the company dime [14:13] Measuring ROI in headcount saved [18:26] Keeping humans in control of AI code [22:26] When Opus and Fable stop speaking English [24:19] Multiplayer refinement sessions with AI [28:16] Magic words that steer the model [32:32] Inside the company Meta harnesses
The World Wild Web & Bots With Buying Power
2026/10/20
An AI slips out of its sandbox. A chatbot won’t explain your cat’s stripes. And your next shopping spree might not involve you clicking anything. This episode asks whether “dangerously powerful” AI is a security concern, a marketing pitch, or a little of both. Then we put agents to work: booking trips, tackling German paperwork, drawing diagrams, and navigating the chaos of online shopping. Plus: what happens to checkout, advertising, and customer data when your best customer is a bot? Chapters:[00:00] Is AI Danger the Ultimate Sales Pitch?[00:43] Powerful Models, Payments, and Big Questions[01:10] Too Dangerous to Release, Too Good to Ignore?[03:19] Why Take the Test When You Can Steal the Answers?[08:36] A Security Failure or a Marketing Win?[11:02] Powerful AI, Forbidden Cat Questions[16:00] AI Only Needs to Find One Unlocked Door[20:52] Welcome to the World Wild Web[22:47] What Happens When You Give AI a Wallet?[26:00] Could ChatGPT Become the Next Amazon?[29:50] Mum Wants a Holiday. Can AI Handle It?[33:38] Can AI Beat German Bureaucracy?[36:28] Claude, Draw My Diagram[39:39] Why Is Connecting Your Email Still So Hard?[44:24] Stop Giving Me Options. Just Buy the Sofa.[46:40] When Bots Shop, Who Owns the Customer?
Agents Acting Safely on Our Behalf
2026/10/15
What happens to the web when the ones browsing it are no longer people? Sarah Drasner, Area Tech Lead at Google, sits down with us ahead of her Agentic Web keynote at AGNTCon + MCPCon North America. She works through that question from both directions: rebuild the web so agents can use it properly, or build agents smart enough to handle the web as it already exists. We get into the split between computer-use and browser-use agents on one side and API and MCP agents on the other. We also watch a real example of the idea in action: an agent pulling speaker photos from Flickr, hunting down company logos, and loading everything into Canva so the human only touches the creative part. From there, Sarah shares the unlikely origin story of WebMCP (and who's to blame for it). We then pressure-test a bold idea: a WebMCP gateway that could cover the huge share of websites that will never ship support on their own. Along the way, we cover the instinct that separates senior engineers, the ability to sense which business problems are actually worth solving. We also talk about how thought experiments cut through a pile of competing priorities, the hacky WebMCP workarounds already showing up in the wild, and why the best agent experiences are the ones that delight customers in ways they never saw coming. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm Sarah Drasner: https://www.linkedin.com/in/sarahdrasner Timestamps: [0:00] Cold open [0:52] The AGNTCon keynote [1:23] Change the web or fix agents [3:35] Browser agents vs MCP agents [5:48] An agent that builds thumbnails [7:03] Keeping the creative work human [9:12] Agent allowances and engineering instinct [13:34] Thought experiments and priorities [15:23] The WebMCP origin story [17:08] A WebMCP gateway for the web [18:53] It works, but it is not ideal [19:44] Hacky WebMCP in the wild
Bridging the Gap Between AI Builders and Domain Experts
2026/10/12
Everyone is racing to build one agent that can do everything. Justin Schroeder, co-founder of Standard Agents, thinks the future looks more like thousands of tiny ones. Ahead of his talk on domain-specific agents at AGNTCon + MCPCon North America, he explains why tightly scoped agents might be the smarter bet, even if it feels like a step back toward the single-purpose machine learning models of a few years ago. We dig into standards as metadata you pin to an agent, so that it shows up with everything it needs in its harness. Then we test the idea against the obvious objection: if a big model can already handle the task, why scope down at all? The answer covers forcing functions, a microservices mindset for agent design, and small agents that quietly check issues or unread messages every five minutes. It also covers the surprise $1K bill waiting for anyone who leaves their agents running. Then it turns into a proper debate. Our host is 99% sure the agent marketplace will never happen, and Justin arrives with a counter that might change his mind (hint: think about where your skills come from). Along the way, we cover agents as tools versus a big-brain router and how anyone keeps track of a thousand agents. We also talk about why the people who know what agents can do and the people who know what they need still have to meet in the middle. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm Justin Shroeder: https://www.linkedin.com/in/jpschroeder Timestamps: [0:00] Cold open [1:03] The future is domain-specific agents [1:51] God models vs specialist agents [2:20] The perfect Gmail agent [3:14] Why standards are missing [3:29] Train the prompt, not the model [4:30] The harness beats the model [6:07] Standards and runtimes for agents [8:17] Why scope down at all [10:07] Why model routing won't win [10:29] The surprise $1K bill [11:12] Agents that build agents [12:09] Agents as tools [13:23] Managing a thousand agents [14:24] Agents for Main Street [15:37] Agent marketplaces will never happen [16:52] Skills are just bad agents [18:04] Skills are already a marketplace [20:12] Domain experts build the agents [22:12] Wrap up
Inside a Game Where AI Agents Never Sleep
2026/10/08
What happens to an MMORPG when half the players never log off? Issac Lee, who runs AI and blockchain experiments at NEXUS, has been testing that idea. His team is filling game worlds with agent players instead of scripted NPCs, so a new title feels alive from day one. We get into why agent-generated content could be the next step after studio-made and user-generated games. We also cover how agent populations might solve the chicken-and-egg problem every multiplayer launch faces, and what it means when your fellow players are relentless and never need sleep. Isaac walks through the NEXUS lab: a text-to-3D world builder for Roblox, an AI pipeline that turns ONE store novels into short web dramas, a loop-based tool for natural pixel animation, and Da Vinci, the internal multi-agent framework the team uses to ship. Then the conversation turns to infrastructure. We talk about why NEXUS went from 200 hand-defined agent roles to a much thinner harness, whether custom harnesses still matter now that Claude and Codex exist, and why the Kimi release pushed the company to start building its own small data center for inference. We close on context: how much of your life you should hand to an AI, what personalized meeting notes could look like, and why shopping agents may force every ecommerce site to rethink who it's designing for. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmIssac Lee: https://www.linkedin.com/in/issacley/?locale=ko Timestamps:[0:00] Cold open[0:57] AI in gaming at NEXUS[2:19] Text to 3D and the last 10 percent[4:38] AI companions and coaches[6:35] Agent players in MMORPGs[8:27] From UGC to agent-generated content[11:07] Agents that never sleep[12:58] Taming fuzzy prompts[19:24] Inside Da Vinci[23:10] Undercover QA agents[27:14] Do harnesses still matter[31:52] Self-hosting open models[41:33] Lab experiments and Bob[50:42] How much context is too much[54:22] Marketing and shopping agents
How a Logistics Giant Keeps AI Data Locked Down
2026/10/05
Picture a startup with ten engineers hammering away on AI and one person lying awake over a $20,000 bill that hasn't arrived yet. That's the scenario we put to Jason Ward, who handles FinOps for AI at C.H. Robinson and recently joined the FinOps Foundation's AI working group. Jason's first answer is not glamorous: tag every AI resource so you know who owns it. The rest of the episode is what that makes possible. He walks through how C.H. Robinson runs AI across order entry, quoting, booking, and tracking, and why the team routes Anthropic models through Vertex AI to keep data locked down. Then come the metrics. Jason uses AI to dig through his own observability platform for signals he didn't know were there. One of them is how chatty a model is. That signal turned a prompt bloat alert into a bug in the code that kept retrying and burning tokens. Its opposite, context starvation, burns tokens too: a model with too little context keeps failing and trying again. We also cover why agentic and conversational workloads need separate baselines. Jason explains why cost per order is the easy win, and why most of the real work doesn't fit into neat discrete tasks. That's where his experimental cost per thought metric comes in, with reasoning ratio and cache hit rate alongside it. His advice is simple: your AI is the best tool you have for understanding your AI. Alex Salkever: https://www.linkedin.com/in/alexsalkever Jason Ward: https://www.linkedin.com/in/jward2 Timestamps:[0:00] Cold open[0:33] Meet Jason Ward from C.H. Robinson[1:18] AI use cases in logistics[1:52] Azure OpenAI and Vertex AI[2:19] Why keeping data in-house matters[2:44] How developers use AI day to day[3:40] Using AI to hack AI observability[4:21] Measuring how chatty a model is[5:45] Advice for a startup afraid of its AI bill[6:33] Step one is tag every AI resource[7:05] The Copilot billing blind spot[7:44] Why spend is only half the story[8:12] Break down cost by app and by model[9:09] The prompt bloat that exposed a bug[10:03] Context starvation[11:28] What goes into the AI spend report[12:21] Discrete tasks are low-hanging fruit[12:58] Agentic vs conversational workloads[14:12] Know the workflow before you report on it[15:02] The CTO's end goal[16:08] Cost per thought[17:59] Reasoning ratio and cache hit rate[19:06] Dynamic model routing[20:04] Use your AI to improve your AI
The Caveman Prompting Challenge
2026/10/01
Caveman prompting has one rule: why use many words when few do the trick? It saves tokens on the way in and on the way out. Push it too far, though, and the output falls apart. So how far is too far? Nobody has benchmarked it yet, and that question opens our conversation with James Barney, Head of Forward Labs at MetLife. James spends his days connecting new AI capabilities to old business problems across dozens of regulatory regimes, and he still finds time to push code. He explains how the FinOps Foundation's AI working group took on the most basic question: which model for which workload, and why the answer always comes down to cost, speed, and accuracy. We get into Anthropic's launch pricing for Fable, why a million tokens is easy to price and hard to explain, and why every stakeholder eventually tells you what they really care about once you name the wrong North Star. From there it gets practical. Start with the smartest model, then step down and add harness until quality holds. Treat exploration tokens like local builds and production tokens like pipelines. Govern agents the way you govern people, with proactive blocks, reactive checks, and policy as code an agent can actually read. We close on a bigger shift. When a chat window can pull from every dashboard at once, do we still need dashboards? James thinks mostly not, with one catch he calls the latent shopper problem: some insights only come from browsing data you did not know to ask about. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkm James Barney: https://www.linkedin.com/in/james-barney Timestamps: [00:00] Cold open [01:03] Meet James from MetLife [01:11] What AI enablement means at a global insurer [03:27] Inside the FinOps Foundation AI working group [05:03] The caveman skill challenge [06:37] Why we need a CaveBench [07:43] Anthropic's Fable launch pricing [09:34] Planning for surprise model releases [11:45] Measuring AI value beyond cost [12:47] Finding your unit metric [14:15] Tying token spend to business outcomes [16:35] Explore first, then optimize [18:12] AI that tunes itself [20:13] Do R&D tokens count [23:24] When the experiment becomes the product [25:38] Personal agents and daily briefings [27:39] Governing agents that run just because they can [30:37] The layers of AI governance [31:56] Proactive and reactive guardrails [32:59] Policy as code for agents [34:41] Stop the click ops [35:31] Is the modern UI obsolete [36:36] MCP apps and chat as the new browser [38:07] How AI gathers data differently than humans [40:37] The latent shopper problem [42:12] Staying close to your data

Podcast reviews

Read MLOps.community podcast reviews


4.6 out of 5
25 reviews
★☆☆☆☆
Thank T. 2026/05/10
Lots of stealth marketing pitches, very little information
The hosts are a couple of young guys who spend most of the time congratulating themselves for moving to the Bay Area. Lots of “everyone who’s serious ...
★★★★★
SatayAsdf 2026/02/11
10 out of 5 stars
Amazing host/podcast, very informative and awesome guests.
★★★★★
Stealth912 2023/08/22
Consistently good information from operators
No fluff. Getting into the details. There's no other podcast like this. Thank you for sharing!
★★★★★
goalieagk 2022/04/04
Interesting discussions covering a rapidly developing field
I’m a senior ML engineer who deals with a lot of MLOps related items since we don’t have a dedicated role for that on our team. This show (and the Sla...
★★★★★
bmorphism 2022/04/03
This podcast is art! Grazie ragazzə 🎉
Long time listener, glad the show is going strong. 🦾 More generative art with ANNs in production, please! Looking forward to applying insights from ...
★★★★★
uber_gadgetz 2021/10/08
Your Guide to Successful MLOps!
Great interviews that tease out everything you need to know regarding the MLOps tooling, best practices, and engineering culture required to execute s...
check all reviews on apple podcasts

Podcast sponsorship advertising

Start advertising on MLOps.community relevant audience podcasts


What do you want to promote?