Advertise on podcast: The Test Set by Posit

Greg Swinehart mocked up the first rstudio.org, and JJ Allaire and Joe Cheng sent it back for looking too good. He's been at it ever since, from hex stickers to Shiny's new look. He and Michael Chow get into the "JJ Allaire patented heat index," why the best design would have no words, and the four-person crew with a broken laptop that redesigned every Safeway. Plus, Michael's 2019 rstudio::conf pecan pie heist. What’s Inside Why RStudio's first website couldn't look like "a real software company"The JJ Allaire patented heat index, and what "too hot" meansHow playing drums in a band turned Greg into a designerShiny apps that match your brand without you thinking about itA four-person crew, a broken laptop, and every SafewayThe CHOP data scientist who made Greg crySpotting AI slop from a long way away

Posit CEO Tareef Kawaf’s theory about LLMs is that they work like an exponent on whoever you already are. Greater than one and you compound. Under one and you accelerate in the wrong direction. Michael, Wes, and Hadley chat with Tareef about what that means for data science, along with John Chambers' Prime Directive, why Tareef now goes up against his own board, and the cat he waited 40 years for. What’s Inside LLMs as an exponent, and why the base is youThe CEO bug hunterJohn Chambers' Prime Directive and why AI makes it matter againA Kaypro 16, 128K of RAM, and an uncle in DaytonaThe 70 percent of your job that stops being yoursThe oud, and the cat he waited forty years for

Leland McInnes has spent his career building tools a lot data scientists take for granted (UMAP, HDBSCAN), and amusingly the applications of those tools keep getting stranger. On this episode of The Test Set, he tells Michael, Hadley, and Wes about teams tracking laundered gold through mineral impurities, and the time he helped casinos optimize slot machine layouts. Also, why nobody's intuition survives contact with high-dimensional space. On this episode of The Test Set, we hit all that, plus imposter syndrome, AI as a sounding board, and the case for asking better questions instead of chasing answers. What's inside Wild UMAP use cases, e.g., catching laundered goldWhy casinos pay mathematicians to rearrange slot machinesHigh-dimensional space and how it defies human intuitionNeural embeddings for people who aren't AI researchersImposter syndrome — across two disciplinesLeland’s take on AI as a rubber-duck testing partner“Look at your data" because it legit beats guessing

Ruth Milligan has coached hundreds of conference speakers, curates TEDx Columbus, and wrote The Motivated Speaker. She's here to tell you that speaking is habitual, not natural. Michael, Wes, and Hadley get a live fifteen-second filler word hack, an uncomfortable truth about listening back to your own voice, and the three kinds of pitches Ruth has been reading for seventeen years. What's inside Why a great idea beats a great speakerSpeaking is habitual, not natural, and that’s goodThe fifteen-second breathing trick to kill fillersHow many charts do you need? Fewer than you thinkThe three pitches Ruth has read for seventeen yearsYou can't get better until you listen to yourselfHadley contends people who send voice memos are monstersMentioned in this episode Patsy Rodenburg's "Three Circles of Energy" — a summary of the frameworkRébecca Kleinberger, "Why you don't like the sound of your own voice" (TED)

Trevor Manz went from measuring plant apertures by hand in a wet lab to building the notebook that lets coding agents take the wheel. The creator of anywidget and founding engineer at marimo (marimo.io/pair) popped into The Test Set to spill on reactive notebooks, why marimo pair threw out every MCP tool but one, and what agents really want out of a data environment. This conversation also features a jacket bouncer, a hidden Python API, and Michael's slow-motion war with the word "marimo." What's inside: Cell order doesn't matter in a reactive Python notebookWet-lab pipettes and Harvard's visualization group, via Raspberry PiThe problem with building beautiful tools nobody actually usesThe reason marimo pair deleted every agent tool but oneCode mode: the hidden API humans aren't supposed to touchWhat happens when you ship the API your LLM hallucinated

Leilani Battle studies how software shapes what we see and believe. The University of Washington professor and co-director of the UW Interactive Data Lab talks with Michael, Hadley, and Wes about an experiment that manipulated people using nothing but loading speed, and why AI models don't seem to recommend charts the way the community that studies charts actually does. Other highlights: rationality's blind spots and a thorough disc golf origin story. What's inside: Loading spinners that quietly change what people findAI models that don't recommend charts like humans doThe blurry line between databases and human factorsA behavior-change experiment for building fairer modelsRationality's role in producing unethical outcomesDisc golf origin storyA vegan pancake recipe

Joe Cheng is the CTO of Posit and the creator of Shiny. He joins Michael and Hadley to talk about why he almost walked away from AI work entirely over ethics concerns and what it takes to lead a team that didn't necessarily choose you. Plus, why saying yes to everyone is a worse strategy than it sounds. Bonus: Hadley calls out Joe's people-pleasing in real time. What's inside: Joe's 2012 self-doubt spiral that accidentally created ShinyWhy Joe almost quit working on AI entirelyThe "loaded guns" problem with releasing AI toolsHadley's blunt leadership style vs. Joe's people-pleasingNobody actually wanted to make Joe CTO?Joe's take on curiosity, duty, and fear as motivators

Caitlin Colgrove is the CTO of Hex, the data workspace for building and sharing data projects using SQL and Python that somehow counts a Sweetgreen chef as a power user. She joins Michael, Hadley, and Isabel to talk about what AI agents actually get wrong in data work (it's not the hallucinations, it's supreme overconfidence), why data teams aren't going anywhere, and how she thinks about building products for humans and agents at the same time. What's inside What Hex's Context Studio does, and why it's a data team's new jobMore code is now written in Hex by agents than by humans"My job is to vouch for the correctness of the answer" — redefining the data teamThe vibe-coded CEO PR is coming for your data team (if it hasn’t already)Soulsborne games as couple's therapy, aka, the Elden Ring co-op report

Alex Hillman built one of America's first co-working spaces, wrote a business book in tweets, and recently handed his inbox to a Claude Code agent — not to draft emails, but to notice when a friendship is going cold. In this episode, Alex, Michael, Wes, and Hadley dig into marketing for people who hate marketing, what 20 years of email reveals about your relationships, and why the hardest part of AI-assisted coding was always before you wrote a single line. What's inside:  Marketing is really just listening at scaleBuilding a 20-year relationship database from your sent folder"Hot rod vs. plumbing" — the two kinds of software you build nowWhat early internet and the AI boom have in commonThe case for reading 20-year-old engineering books with a coding agentKaraoke philosophy as a framework for community building

Mike Bostock made D3 when the browser was still a joke. He built bl.ocks when people needed somewhere to share their work. Now he's building Observable — reactive notebooks with an AI that actually looks at what it made. In this episode: the three-GIF bar chart that launched 25 years of viz, why open source needs both intrinsic and extrinsic motivation, and why an agent that can't see its own output is likely to be confidently wrong. What's Inside The 1998 visualization library that could only make bar chartsWhy D3 hit #3 on GitHub, and what killed the galleryWhat spreadsheets got right that notebooks ignored for years"The agent can lie with text, but not with code"Why Observable scrapped canvases and went back to notebooksThe penguin dataset that exposes AIStrength training, tennis mind games, and a resurrected Stanford game

Paige Bailey is a developer relations engineering lead at Google DeepMind. She's a geophysicist-turned-AI-engineer who was once told by her professors that building open-source libraries was a waste of time. We talk about her path from planetary science to TensorFlow, why statisticians have a hidden edge in the age of AI, and what it means to be a curious generalist when the cost of building software is approaching zero. Bonus: installing solar-powered silent-film birdhouses as street art in San Francisco. What's inside From planetary science to TensorFlow, before it was GPU-capableGeophysicists as early GPU adoptersThe professors who said open-source wasn’t “real science”Building silent-film birdhouses as San Francisco street artHiding Gemini API tests inside whimsical side projectsThe right-tool-for-the-job case for mixing AI modelsWhy “taste” is the skill that matters when code costs nothing

Vincent Warmerdam has been the first full-time hire at a startup, a spacey punster who accidentally got himself a job, a bartender at an Amsterdam comedy theater, and a Dutch bike tour guide — and he'll tell you all of it was career development. Now doing DevRel at Marimo, Vincent makes the case for reactive notebooks, Lego-brick widgets, and why "number go up" is not a data science strategy. Also: chickens die. The model doesn't know. This matters more than you think. What's inside How a spacey pun accidentally launched Vincent's careerWhy Marimo's constraints make it better for LLMs, not just humansThe gorilla hiding in your dataset — and why the model missed itVibe coding vs. notebooks: three cells at a time as a disciplineWidgets as Lego bricks: reusable, composable, criminally underusedCognitive debt, confirmation bias, and sycophantic data scienceWhy natural intelligence is still, actually, a pretty good idea

Benn Stancil built Mode Analytics, spent a decade in the data trenches, and now writes some of the sharpest, funniest essays in the data world. On The Test Set, he talks about the cultural shift from Nate Silver to Rick Rubin why AI might kill the analytics dashboard, and what happens when a thousand startups all build the same thing. Plus: boy bands as a model for collaboration, and why the best creative work starts with cheating. What's inside:  Why the modern data stack was basically big data 2.0The cultural flip from Nate Silver to Rick RubinGas Town, tar pits, and the AI startup zero-sum gameSoftware is becoming content, and that changes thingsBenn's creative process: Lorde lyrics, Codenames, and cheatingThe boy band as a model for small-team collaborationBI is (mostly) dead, and vibes might replace SQL

dbt Labs CEO Tristan Handy drops into The Test Set to map the fault lines between the data science world and the enterprise data world — and explain why analytics engineers are basically pissed-off data analysts who decided to organize the bookshelf. We get into SQL's glow-up, the community magic of dbt Slack, what AI agents mean for data warehouses, and why everyone's building iOS apps with Claude now. What's inside: What analytics engineers *actually* doSQL's journey from deeply unsexy to indispensableHow dbt turned source control into a source of truthBuilding a tech community without the RTFM energyAI agents on your data lake: permissions get personalWill LLMs kill the open-source package ecosystem?Edible gardening, welding dreams, and digital dysphoria

Nell Thomas has spent two decades in data — from equity research to the DNC to Facebook to leading a 400-person data org at Shopify. She walks Michael and Wes through the modern data stack role by role, gets honest about what AI is and isn't changing about data work, and admits the semantic layer has been her greatest leadership failure. Plus: Sneakers gets the respect it deserves. Episode Notes What does it actually look like to run data infrastructure for millions of merchants while the entire industry reinvents itself in real time? Nell Thomas (VP of Data, Shopify) talks vibe-coded dashboards, political campaign data scarcity, blameless postmortems, and why no one should be locking in on an AI strategy just yet. Recorded live in Times Square. What’s Inside Mapping the modern data stack, role by roleWhy data quality is still the #1 problemWhat "good scrutiny" looks like on a data teamVibe coded dashboards and the trust problemShopify's MCP for their data warehouseThe throwaway tech problem in political campaignsWhy the semantic layer is so damn hardSneakers!