Advertise on podcast: The Information Bottleneck

In this episode, we talked with Daniel Litt, a mathematician at the University of Toronto. In March 2025, Daniel bet that AI would not be able to autonomously produce an Annals of Mathematics-level paper by 2030. He has since conceded. He told us what changed and why. The models became superhuman at a small set of skills, like knowing the entire literature and grinding through technical arguments, and that turned out to be enough to do real math. They are still weak at the fuzzy part of the job, like figuring out what is actually true about an object or which questions are worth asking. We also got into how anyone can check a proof that no human has read yet, and what a Lean formalization does and doesn't guarantee. Daniel thinks prestige and credit in math are dead, and that we will have to judge people through thesis defenses and seminars rather than through PDFs. He explains why he thinks math got Goodharted and why the labs put so much effort into it in the first place. We finished with listener questions, including how to teach kids math now and whether P vs NP might be just one idea away. Topics - Why Daniel conceded his AI math bet - What models are superhuman at in math, and where they fall short - Why AI won't "solve math" - Checking AI-written proofs and the limits of Lean - Why different models keep proving the same results - How Daniel uses AI in his own research - Evaluating mathematicians when prestige and credit are dead - Why math got "Goodharted" - AI labs, race dynamics and publication norms - Teaching kids math, and P vs NP 00:00 Intro 00:15 The bet Daniel conceded 03:23 What AI is good and bad at in math 08:13 Will AI solve all of math? 11:37 Why we still need human mathematicians 13:59 Math is about understanding, not only proof 21:26 Axioms, foundations and Lean 27:42 How do you check an AI-written proof? 31:22 When models cheat 36:18 Are all the models proving the same things? 39:51 How Daniel uses AI in his own research 45:20 "Prestige and credit are dead" 49:58 AI risk and social upheaval 51:38 Is math special, or did it get Goodharted? 57:45 Why the labs care about math 59:54 PR, race dynamics and the pause 1:06:49 Would Daniel join a lab? 1:07:53 Listener questions: can understanding be formalized? 1:12:05 Teaching kids math with AI around 1:15:38 P vs NP and the Riemann hypothesis 1:17:37 Wrap-up Music: "Kid Kodi" by Blue Dot Sessions (CC BY-NC 4.0)

In this episode of the the information bottleneck podcast, we talked with Andy Hall, a Stanford professor and MTS at Anthropic. The conversation explores the political economy of AGI, analyzing how superintelligence could reshape job markets, wealth redistribution, and public trust in government. Hall presents an optimistic view that hyperintelligent agents could enhance democracy by helping citizens make better political decisions, likening AI's potential impact to the invention of the printing press. The discussion also covers the challenges of regulating open-source models, the origins and mitigation of ideological biases in AI, and the geopolitical implications of the AI race between the US and adversaries like China. List of TopicsThe definition of the political economy of superintelligence and its potential to improve civic decision-making.The future of work, the likelihood of white-collar job loss, and the role of the existing tax system in wealth redistribution.Public skepticism toward AI, the decline of trust in the US government, and the contrast between AI power users and the general public.The tension between personal freedom and collective safety regarding open-source and open-weight AI models.Using AI tools to assist legislators in drafting policy and auditing frontier AI models.The geopolitical AI race, the threat of state-run hacking groups, and the challenges for middle-power countries.Center-left ideological biases in current AI models, their roots in training data and reinforcement learning, and experiments to fine-tune models with alternative viewpoints.The concept of AI constitutions, market competition over model values, and the risks of government intervention.AI-driven astroturfing on social media, ungovernable AI swarms, and the debate over regulatory "kill switches".Timeline00:14 - Introduction of Andy Hall and an overview of the political economy of superintelligence.05:38 - Discussion on the future of work, potential mass unemployment, and the political backlash of job loss during periods of economic growth.11:41 - Analysis of public hostility toward AI, drawing analogies to the regulation of social media.19:26 - The role of existing tax systems in wealth redistribution and the political dynamics of state versus federal taxation.27:31 - The debate over open-source models, balancing the desire for personal AI agents with national cyber risk concerns.33:10 - How AI can aid government bureaucracy, specifically in writing complex legislation and overseeing tech labs.46:25 - The geopolitical implications of AI, the race against China, and the threat of state-run hacking organizations like the Lazarus Group.51:38 - Examination of political biases in AI models, including Hall's lab experiments in fine-tuning models to hold conservative views.1:02:38 - Discussion on model constitutions, the distinction between a "soul document" and a true political constitution, and the regulation of AI values.1:13:19 - The rise of AI astroturfing in politics, botnets, and whether "kill switch" legislation is a practical solution.1:19:40 - Theoretical discussions on the future of automated law enforcement, robot armies, and AI's impact on class mobility.Music - "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

John Platt is a Google Fellow and leads the Applied Science team at Google Research, the group working on climate and science. He's also the person behind SMO and Platt scaling. In this episode, we talk with him about how Google decides which scientific problems are worth its time, and what happens when AI meets real physical systems. A big part of the conversation is contrails. A small share of flights creates most of the warming from contrails, which together add up to roughly 1% of human-caused warming. John explains how his team detects contrails from weather satellites and predicts where they will form. In trials with American Airlines, rerouting flights cut contrail formation by 62% with almost no extra fuel. We also get into ERA, Google's empirical research assistant, and where AI agents actually help scientists. John sees them taking over the drudgery of building models and munging data, while humans keep the taste and the rigor. He shares why he's skeptical of gradient-free optimization, why the bitter lesson runs into energy and data center limits, and why physical AI that makes manufacturing cheaper could matter a lot for climate. We finish with FireSat, a satellite constellation (about 50 satellites at full scale) designed to catch wildfires while they're still small enough to put out. Timeline 00:00 Intro 01:09 How Google picks science problems: comparative advantage 04:15 Projects that didn't work: RNA aptamers and tool/problem fit 05:46 DNA, RNA and XNA 07:41 Contrails and why they warm the planet 13:01 Three AI systems for avoiding contrails 15:23 Cloud seeding and why cloud physics is hard 18:17 Airline trials: 62% fewer contrails 19:46 Navier-Stokes, wind tunnels and where simulation still fails 25:07 Will agents replace scientists? ERA and scorable tasks 31:36 Overfitting, Kaggle and the half-pixel bug 33:41 Agents that exploit bugs: Lean and Goodhart's law 36:45 Gradient-free optimization 38:53 SMO, SVMs and hinge loss in neural networks 42:12 The bitter lesson and the limits of scaling 43:36 Where agents will help science in the next 12 months 45:13 Robotics and making climate tech cheaper to build 48:22 AI risk and misuse 50:51 Climate outlook: past 2°C 54:39 Why AI won't just solve climate change 57:04 Room-temperature superconductors, fusion and HVDC 1:02:25 FireSat: detecting wildfires from orbit 1:05:16 Why existing satellites aren't enough 1:07:53 The economics of 50 satellites 1:09:34 False positives 1:11:16 Closing Topics Comparative advantage as the test for which science problems Google takes onContrails: the physics, the AI pipeline, and the American Airlines trialsERA and AI agents for scientific softwareWhat scientists still do that agents can’t: taste, creativity, rigorOverfitting, Goodhart’s law, and agents exploiting bugsGradient-free methods, SMO, and hinge lossThe bitter lesson and the physical limits of scalingClimate outlook, fusion, and physical AIRoom-temperature superconductors and long-distance power transmissionFireSat and early wildfire detectionMusic - "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Yuandong Tian spent many years at Meta FAIR and recently left to co-found Recursive Superintelligence, a company building AI that improves itself. In this episode, he tells us why. We start with his work on computer Go, where he built DarkForest before AlphaGo came out. A few years later came OpenGo, which played Korean professionals on a single GPU and didn't lose a game. He then tried to bring RL to real-world problems and found that the design of the action space mattered more than the algorithm. We also talk about Coconut, his paper on reasoning in latent space instead of tokens, and why frontier models still aren't trained that way. The second half is about recursive self-improvement. Yuandong wrote his last paper at Meta together with GPT-5 and says it made him 6 to 10 times faster. That convinced him his own job could be replaced within five years. We ask him where agents still fall short and whether a new architecture can really beat transformers at scale. He also gives his view on the calls to restrict self-improving AI and makes the case for open source. Recursive is hiring in San Francisco and London: [email protected] Topics * Computer Go: DarkForest, AlphaGo and OpenGo * Gradient-free optimization * Action space design and neural architecture search * Coconut and reasoning in latent space * Understanding how neural networks learn representations * Grokking, and writing a paper with GPT-5 * Recursive self-improvement and coding agents * New architectures vs. transformers * The NanoGPT speedrun * Data efficiency and robotics * Restricting self-improving AI * Open source models Timeline 0:00 Intro 0:45 From CMU to deep learning, and the AlexNet debates 5:35 AlphaGo and beating Go pros on a single GPU 10:27 Gradient-free optimization 13:16 Diffusion models and diversity 16:23 RL on real problems: why the action space matters 20:34 AutoML and architecture search 22:05 Coconut and reasoning in latent space 26:07 Why latent reasoning hasn't caught on 30:36 Adapting research to the LLM era 34:04 Why he left Meta to work on recursive self-improvement 37:10 Do we still need humans? 39:37 Where agents fall short 43:08 Can new architectures beat transformers at scale? 47:29 The hardest part of research to automate 50:16 What comes next for AI 52:52 Should self-improving AI be restricted? 54:11 Open source models 56:43 Hiring at Recursive Music - "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Chris Manning is a legend in NLP. He's a professor of linguistics and computer science at Stanford and ran the Stanford AI Lab. His course CS224N, Natural Language Processing with Deep Learning, is where a whole generation of researchers learned the field. If you've worked on anything in NLP over the last 25 years, you've probably used his work. We start with what linguistics gave machine learning that ML wouldn't have figured out on its own, and what's left for NLP researchers now that LLMs handle most of the classic tasks. Chris thinks the open problems have moved up the stack to pragmatics and dialogue. Models are good at using context but still sound confident when they shouldn't. Then we get into his recent work on how language models learn verb classes. Small GPT-2 models seem to form abstract categories right away instead of memorizing verbs one at a time, and Chris explains why distributed representations push them in that direction. The second half is about meaning and representation. Chris makes the case that Yann LeCun underrates the role of language in intelligence, and revisits his debate with Emily Bender over whether text alone can teach meaning. We finish with ReFT, which steers a frozen model by editing its hidden states, and whether concepts really live in linear subspaces. Timeline 00:00 Intro 01:05 What linguistics gave machine learning 03:27 Is NLP losing its focus on language? 06:29 What's left for NLP researchers in the LLM era 09:37 The next frontier: pragmatics and dialogue 13:01 Constructed languages and AI-to-AI communication 15:35 Do language models learn categories first? 23:43 Why transformers learn abstractions early 26:37 Does it hold at scale? 28:30 LeCun, JEPA and the role of language in intelligence 34:54 Can text alone teach meaning? The octopus debate 40:45 Diffusion language models vs transformers 43:39 ReFT: editing representations instead of weights 50:34 Weights or representations: where knowledge lives 52:37 Do concepts really live in linear subspaces? Topics NLP, computational linguistics, large language models, learning dynamics, meaning from form, world models, JEPA, diffusion LMs, ReFT, representation learning, interpretability Music "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Andrew Dai spent over a decade at Google Brain and DeepMind, where he co-wrote the 2015 paper that introduced language model pre-training followed by fine-tuning, and later co-led pre-training data for Gemini. He's now co-founder and CEO of Elorian AI, which is building models for visual reasoning. We talk about how his pre-training result started as a bug, why next-token prediction scales better than other objectives, and what went wrong for Google in the early LLM race. The second half is about vision: why today's frontier models still can't count objects in a photo, why he thinks reasoning is fundamentally visual, and how his view of world models differs from JEPA. Chapters 00:00 Intro 00:48 Andrew's background and the accidental discovery of pre-training 07:10 Why next-token prediction scales 10:09 How Google fell behind and the early days of Gemini 18:28 What makes training data good 28:16 Where visual understanding breaks down 46:01 World models, JEPA and robotics 55:27 Generation vs understanding, and what's next for Elorian Topics Pre-training and fine-tuningScaling and next-token predictionGemini and Google's AI historyData quality and synthetic dataVisual reasoning and countingWorld models and JEPAMusic "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Zachary Lipton is an associate professor at Carnegie Mellon University and a co-founder of Abridge, a healthcare AI company (and a jazz saxophonist!). His research spans machine learning, healthcare, and the broader impact of AI. In this episode, we discuss what AI can (and cannot yet) do in healthcare. We talk about why medicine is harder to automate than coding, AI scribes and clinical decision support, drug discovery, whether AI could eventually replace doctors, and the role of open models and intelligent routing. In the second half, we turn to the future of AI research: whether academia can still compete, what AI PhD students should work on, and how Zach thinks about automation and the future of research. Timeline00:00 — Introduction 00:28 — Why healthcare is hard for AI 05:00 — Zach’s path into healthcare AI 22:13 — Where AI can have the biggest impact 32:01 — Can AI replace doctors? 41:41 — Open models and AI infrastructure 56:20 — The future of AI research 1:03:03 — What should AI PhD students work on? 1:27:00 — Advice for researchers and founders TopicsAI in healthcare • AI doctors • drug discovery • clinical decision support • open-source AI • AI research • academia vs. industry • future of work Music "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

What comes after scaling? We talk with Sara Hooker, co-founder and CEO of Adaptation Lab, about why the next generation of AI may look very different from today's static models. Sara argues that models should continuously adapt to new tasks, data, users, and environments—and that doing this efficiently will require rethinking much more than fine-tuning. We discuss continual learning, AutoScientist and automated research, why non-verifiable tasks may become the next major bottleneck, and why interfaces could be as important as the models themselves. We also get into open vs. closed models, distillation and Chinese AI labs, AI regulation and safety, cybersecurity and biorisk, AI companionship, and what may eventually come after Transformers and tokenization. TopicsContinuous learning and adaptive AIFine-tuning, memory, and AutoScientistAI agents and automated researchNon-verifiable tasks and human feedbackAdaptive interfacesOpen vs. closed models and distillationAI safety, regulation, cyber risk, and bioriskAI companionship and persuasionThe limits of TransformersMultilingual models and tokenizationChapters00:00 — Introduction 02:15 — Why start another AI lab? The return of research 05:46 — What continuous learning actually means 12:04 — Should every company have its own adapting model? 13:59 — Fine-tuning and platforms like Tinker 18:04 — AutoScientist and automated optimization 22:52 — Can AI really improve its own research? 28:38 — The problem of non-verifiable tasks 31:30 — Human feedback and the limits of exponential progress 34:43 — Why the AI interface matters 40:36 — Distillation, China, and open models 49:05 — Open-model licensing 52:19 — Will open models catch closed models? 58:43 — AI regulation and compute thresholds 1:03:07 — AI safety and agent failures 1:10:19 — Biorisk vs. cybersecurity 1:14:03 — Persuasion, AI companionship, and overlooked risks 1:20:41 — Where will AI have the biggest real-world impact? 1:25:41 — What is missing from current AI architectures? 1:29:03 — Neurosymbolic AI 1:31:30 — Multilingual models and tokenization 1:34:02 — Byte-level models and alternatives to tokenization 1:35:03 — Closing Music "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Alexia Jolicoeur-Martineau is a Principal Researcher at Microsoft and the author of "Less is More: Recursive Reasoning with Tiny Networks," the paper behind the Tiny Recursive Model that hit about 45% on ARC-AGI-1 with a fraction of the parameters of frontier systems. It won the 2025 ARC Prize paper award. She read the hierarchical reasoning paper, thought the potential was real and the explanation was not, and rebuilt it without the mouse brains: a small network that carries a hidden state and a current answer, thinks for a few steps, updates, and repeats, with the gradient truncated at each loop. We get into why puzzles suit this and autoregression doesn't, why she thinks LLMs are bad at molecules and more data won't fix it, and what she'd do with a trillion dollars. Timeline 00:01 Intro01:06 Leaving biostatistics, and why the field stagnated06:47 GANs, diffusion, and research on four GPUs12:58 What was wrong with the hierarchical reasoning paper16:31 Tiny recursive models explained without the biology22:35 Why puzzles favor recursion over left to right generation24:15 Is the bitter lesson really bitter?27:28 With infinite compute, would you still want small models?32:00 Self improvement, memory, and a trillion dollars37:01 Test time compute beyond chain of thought40:41 Why chain of thought fails on molecules45:17 Is there a universal representation?48:06 What people are already building with TRM55:22 Fixed point models and DEQMusic "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0Topics Tiny Recursive Models and the ARC-AGI resultsWhat the hierarchical reasoning model was really doingDeep supervision and truncated backpropLooping transformers and parameter efficiencyWhy puzzles favor whole-context iteration over left to right generationTest time compute beyond chain of thoughtLatent reasoning and the Coconut line of workWhy LLMs fail on chemistry and physicsRepresentation learning and whether a universal representation exists

Rohan Anil spent eleven and a half years at Google, where he went from writing memory allocators to large-scale linear solvers, then optimization at Google Brain, where he co-developed distributed Shampoo and led optimization for PaLM and Gemini pre-training, including the work that produced Gemini Flash. He then joined Anthropic's pre-training team, and left before the IPO to co-found Core Automation with Jerry Tworek (ex-VP of Research at OpenAI). We talk with him about how Brain worked at its peak, why he left two of the world's best labs, and what he thinks is missing from today's models. Rohan's view is that pre-training and RL were split by organizational convenience rather than by science. Pre-training builds a prior, and RL sharpens it to the tasks we care about, and neither gives a model a way to absorb new data or learn from its own experience once it is deployed. Post-training more every day plateaus, on-policy distillation plateaus, and in-context learning only goes as far as the context does. He argues the next architecture needs better ways to fold in new knowledge at inference time, and that this is a fundamental optimization question rather than a harness-engineering one. We also get into why coding agents still fail on low-level systems work, his take on Muon, why second-order methods matter once you leave the noise-dominated regime, and why nobody can yet use a few million GPUs for a single training run. Timeline 00:00 Intro 01:09 From computer vision to Google systems engineering 02:37 Large-scale linear solvers and sparse features 06:01 Getting into optimization: SDCA and Yonghui Wu's team 07:29 Joining the Shampoo crew 09:35 The Google Brain ethos, and why 2017 to 2019 was special 14:23 Is open research going to keep winning? 15:53 Frontier models are only as good as the prior you give them 17:30 Missing the language model wave, then Common Crawl and online distillation 18:31 Paternity leave, DALL-E Mini, and the 14 days that became two years 20:32 PaLM, Gemini pre-training, and Gemini Flash 23:59 The Shampoo origin story: Tomer Koren's two-week proof 26:54 Why leave Google for Anthropic 30:20 Why leave Anthropic for a startup 31:33 Meeting Jerry Tworek at Dolores Park 34:00 What Core Automation is building 36:17 Continual learning and the pre-training vs RL split 39:11 Why coding agents fail at kernels and low-level pipelines 42:00 The QR factorization kernel competition and reward hacking 44:57 Numerics, verification, and hardware that keeps changing 46:07 Are LLMs creative, or just good at search? 49:50 Getting models to extrapolate instead of interpolate 52:03 Why did we ever call it pre-training? 55:02 What RL is really learning 56:27 Competing with the big labs with fewer people 58:31 Will kernel generation keep old GPUs alive? Amdahl's law 1:01:58 Open source plans 1:02:55 Audience question: agentic optimizers 1:04:53 Audience question: Muon, Shampoo, and the future of second-order methods 1:08:58 Hiring at Core Automation key topics Journey from Google Brain to startupEvolution of AI research and optimizationPre-training and reinforcement learningKernel optimization and system efficiencyOpen source AI and collaborative researchChallenges in AI creativity and explorationFuture directions in continual learning and model scaling Music"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

John Langford, one of the heads of Microsoft's AI Labs, the creator of Vowpal Wabbit, and a co-inventor of CAPTCHA, joins us to talk about world models. Transformers need orders of magnitude more data than humans to learn the same thing, and John argues a compact, implicit world model is how you close that gap. He explains why he's skeptical of JEPA-style objectives, why a transformer's KV cache is the Ptolemaic epicycle model of belief states, and what his Next Latent work does differently. We also get into whether research still matters in the age of scale; open versus closed models; agent-driven research after running 2,000 pre-training experiments in 90 days; the origin story of CAPTCHA; and why Muon and orthonormal optimizers actually work. Topics: Implicit vs. explicit world models, and the case against JEPA-style objectivesCompact belief states: why compression beats a growing KV cacheDoes research still matter in the age of scale? The Kimi K3 argumentAgent-driven research: 2,000 pre-training experiments in 90 daysThe invention of CAPTCHAOptimizers from SGD and Vowpal Wabbit to Muon Chapters00:00 Why world models: the sample-complexity gap09:48 The case against JEPA; a transformer-style implicit world model15:52 Compact belief states: epicycles vs. heliocentrism23:41 Does research still matter? The Kimi K3 argument27:35 Open vs. closed models35:57 Recursive self-improvement and agent-driven research42:30 2,000 pre-training experiments in 90 days; weak baselines and reproducibility54:54 The invention of CAPTCHA1:00:53 Optimizers: from Vowpal Wabbit to MuonMusic "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Description Tabular data is still where most of machine learning actually happens in industry, and the field has changed a lot in the last few years. In this episode we talk with David Holzmüller, a researcher at INRIA and one of the people behind TabArena, TabICL and RealMLP, about what the state of the art looks like right now and how to pick a model for your own data. We cover the shift to TabPFN-style foundation models that learn to learn from whole tables, why TabArena was built and what earlier benchmarks got wrong, what Google's new TabFM means for the leaderboard, and when gradient boosted trees are still the right tool. David explains why LLMs struggle with tables, shares an early result comparing Claude Opus against TabICL on tiny datasets, and walks through how to embed text columns for tabular models. We also get into time series vs tabular data, the open research problems he thinks matter most, and why classical ML libraries are so bad out of the box. Links: TabArena: https://tabarena.ai Topics Tabular foundation models and in-context learning on tablesTabArena and Beyond Arena: building a benchmark that stays honestTabFM, TabPFN, TabICL and the tradeoffs between themWhen boosted trees and MLPs still win (large data, CPU, fast inference)Why LLMs are inefficient on tabular data and where they might helpEmbedding text columns with language modelsExplainability, calibration and class imbalanceTime series vs tabular dataOpen problems: invariances, synthetic data, uncertainty, scaling downWhere the field is heading in the next five years Chapters 0:00 Intro 0:31 What changed in tabular ML: TabPFN-style foundation models 2:22 Which model to try first? TabArena and how it was built 5:14 What older benchmarks got wrong, and Beyond Arena 8:45 GPU AutoML vs foundation models 10:40 Reading the leaderboard: TabFM, TabPFN, TabICL and the tradeoffs 12:47 Calibration, class imbalance and small vs large data 19:45 Explainability for black-box tabular models 21:34 Why LLMs are bad at tabular data 25:39 Claude Opus 4.6 vs TabICL on tiny datasets 27:51 New classifiers, five-year outlook, real vs synthetic pretraining 33:13 Embedding text columns for tabular foundation models 36:13 Time series vs tabular data 39:59 When gradient boosted trees still win, and feature engineering 45:31 Open research problems and where the field is heading 52:54 Better MLPs and why classical defaults are bad out of the box Music"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Charles Frye (Modal, ex-Weights & Biases, Berkeley PhD) joins Ravid and Allen to explain why modern AI research is bottlenecked by compute, and why simply buying more GPUs doesn't solve it. We cover the three problems every lab hits (underutilization, saturation, resource sharing), when companies should actually train their own models, why inference is a "bad algorithm" for today's hardware, NVIDIA's monopoly, the OpenAI/Hugging Face hack and what it says about open models, and whether we're in a compute bubble. Key topics AI infrastructure challenges and when to train your own modelsGPU resource management and virtualizationInference optimization and speculative decodingThe economics and future of AI hardwareAgents, sandboxing, and open-model security Chapters 00:00 Intro 01:03 Why AI needs special-purpose compute 03:22 Buying vs renting GPUs: the three problems 07:15 Modal's approach, and doing more with less compute 09:46 Do we actually need to spend more? The conflict-of-interest question 13:08 Should companies train their own models? 14:47 Efficient fine-tuning and prompts as fast weights 17:37 Are we in a compute bubble? 20:21 Why inference will dominate compute (the SQLite analogy) 22:42 Speculative decoding 26:44 Why scaling inference is hard, and neuromorphic hardware 28:36 Why NVIDIA's monopoly persists 33:09 Inference chip startups and the hardware lottery 35:24 How Modal stays hardware-agnostic (GPU snapshot restore) 38:45 Will agentic coding erode CUDA's moat? 41:18 Running one agent vs thousands: sandboxing at scale 46:27 The OpenAI/Hugging Face hack and open models as defenders 52:28 Rogue AI, self-replication, and fast takeoff 56:09 What's next: evals, embodiment, edge inference 1:00:27 Modal is hiring (modal.jobs) Music"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0

Stella Biderman, Executive Director of EleutherAI, joins us the week an OpenAI model autonomously broke out of its sandbox and hacked Hugging Face. Stella calls it what she thinks it is, an offensive cyber operation, and argues it's part of a pattern: this is not the first containment failure at a frontier lab, and sandboxes have failed basically every time they've been tested for real. So we spend a good chunk of the episode on what actual containment would look like. Stella's argument is that the tools already exist, the labs just don't use them: run dangerous capability evals on air-gapped networks with no route to the public internet, put the most sensitive testing in SCIF-style secure facilities, and treat model evaluation the way the security world treats classified systems rather than the way startups treat staging environments. And yet Stella remains one of the world's most prominent open-source advocates. From her perspective, the biggest risk isn't the technology; it's unchecked corporate power, and the only durable check on it is an independent scientific research establishment that doesn't depend on the AI industry for its funding or its facts. From there the conversation spans the geopolitics of Chinese open models and whether governments can restrict them, sovereign AI and what it would actually take for other countries to train their own models, why harnesses and UX drive more of AI's perceived progress than raw intelligence, the AI-found counterexample to the Jacobian conjecture, and EleutherAI's "Deep Ignorance" approach to making open-weight models safe by filtering hazardous knowledge out of pretraining. key topics AI governance and regulationCybersecurity incidents involving AI modelsOpen source AI safety and securityThe role of independent research in AI safetyLegal and ethical considerations in AI developmentTimeline 00:13 — Intro: Stella Biderman and EleutherAI, a real non-profit in AI02:05 — News of the week: Kimi K3, and OpenAI's model autonomously hacking Hugging Face05:49 — "Frontier labs can't be trusted": repeated containment failures, air-gapped networks and SCIFs vs. sandboxes22:45 — Can governments ban open or Chinese models? Import restrictions and the six-month open/closed gap27:05 — Why Stella is still pro-open-source: unchecked corporate power as the real danger31:11 — The opioid epidemic analogy: avoiding both regulatory failure and overcorrection34:57 — Offense vs. defense: why open access to AI has empirically favored defenders37:28 — Chinese labs, the CCP, and why safety and fine-tuning are low-prestige work in China42:19 — Sovereign AI: does every country need its own foundation model?49:29 — Sampling, harnesses, and why ChatGPT was really a UX breakthrough54:09 — AI solves the Jacobian conjecture: domain data beats raw intelligence58:02 — Safety is contextual, not a model property — and what HAL 9000 got right1:01:42 — Is Stella optimistic about the future?1:02:50 — Deep Ignorance, the science of AI training dynamics, and how to get involved with EleutherAIMusic"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.

In this episode, Frank Hutter joins us to talk about TabPFN and why tabular data is suddenly the hottest problem in deep learning. Frank is a professor at the University of Freiburg and spent 15 years building the AutoML field before founding Prior Labs, which SAP just acquired for over a billion dollars. We get into why deep learning failed on tables for a decade and what in-context learning changed, how TabPFN is trained entirely on synthetic data, and why a model that never saw a real time series ended up beating specialized forecasting models. Frank also explains the architecture tricks behind scaling from 10,000 to a million rows, where LLMs fit into data science (and where they embarrassingly don't), and what happens to XGBoost from here. Beyond the research, Frank talks about the jump from professor to co-CEO, why he refused to merge his 45-person team into SAP's 110,000 employees, the open-weights licensing debate, and the case for building a frontier lab in Freiburg rather than San Francisco. key topics The role of foundation models in tabular dataImpact of SAP acquisition on Pro LabsThe evolution of AutoML and hyperparameter optimizationChallenges and solutions for large context in modelsOpen source models and licensing strategiesThe importance of independence for startup agilityFuture directions in AI for science and medicine00:00 Intro 00:34 The SAP acquisition and staying independent 07:39 Why tabular data is the next big thing in deep learning 14:19 What makes tabular data hard 19:14 AutoML, AutoGluon, and fifteen years of hyperparameter tuning 28:27 Scaling TabPFN: context limits and architectures 34:35 Agentic data science and LLMs 39:30 Online learning, time series, and Bayesian inference in a forward pass 47:05 Open weights and the license debate 54:51 Will LLMs and tabular models merge? 1:00:01 From academia to startup 1:09:42 Why build in Europe 1:12:53 Audience questions and hiring Music"Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0.