1751366208
Doom Debates

Advertise on podcast: Doom Debates

Rating
★★★★☆
4.1
from
25 reviews
This podcast has
201 episodes
Language
English
Publisher
Liron Shapira
Explicit
No
Date created
2024/06/10
Latest episode
2026/11/13
Average duration
96 min.
Release period
6 days

Description

It's time to talk about the end of the world! lironshapira.substack.com

Unlock Doom Debates podcast Email contact info,
Listeners & Audience details

Email contact information

Direct podcast contact details

Listeners

Audience numbers & engagement insights

Audience details

Podcast Insights

Podcast episodes

Check latest episodes from Doom Debates podcast


Jensen Huang Resigns, RSI Coming in 2027 — with Robert Wright
2026/11/13
Bestselling author Robert Wright and I react to Jensen Huang’s resignation from Nvidia, METR’s report that AI is doubling Anthropic’s research speed, and whether AI safety can ever be formally verified. Plus: Where AI’s economic value will accrue; My AI outperforming my CPA; What Jensen’s fall means for Anthropic’s IPO; And an anonymous Chinese AI journalist on how it all looks from Beijing. Enjoy another edition of the Nonzero x Doom Debates collab! Watch on YouTube: https://www.youtube.com/watch?v=uZZEfe2OmAE Timestamps 00:00:00 — Cold Open 00:00:30 — Introductions 00:01:32 — Jensen Huang Resigns from Nvidia 00:05:02 — Who’s Defending Jensen? 00:10:09 — What Did Jensen Do Wrong? 00:12:23 — Was Alice Huang a Chinese Spy? 00:18:03 — Does Decapitation Work? 00:21:34 — The Great Man Theory of Tech 00:23:58 — Where Will AI’s Value Accrue? 00:27:14 — What’s Next for Jensen? 00:29:33 — METR: AI Is Doubling Anthropic’s Research Speed 00:35:01 — AI Is Solving Open Math Problems 00:37:24 — AI Agents Negotiating Deals 00:40:03 — My AI vs. My CPA 00:42:38 — Bob’s AI Caught His Doctor’s Error 00:44:20 — Whatever Happened to Hallucinations? 00:45:35 — Formal Verification and Lean 00:50:47 — Andrew Wiles and the Human Cost 00:54:10 — The All-In Podcast’s Blind Spot 00:57:15 — Can We Formally Verify Alignment? 00:59:55 — Yudkowsky vs. Irving at ILIAD 01:01:21 — Will Jensen’s Fall Help Anthropic’s IPO? 01:06:48 — The March for an AI Treaty 01:08:02 — Introducing Walter, a Chinese AI Journalist 01:09:22 — How China Reacted to Jensen’s Resignation 01:15:43 — Chinese Theories About Anthropic’s Role 01:22:47 — How China Censored the Spy Allegation 01:24:33 — Does China Trust an AI Pause? 01:26:59 — AI Layoffs and a Chili Sauce Boycott 01:32:50 — Does Beijing Listen to Public Opinion? 01:36:42 — China’s Limits on AI Companions for Kids 01:39:29 — Will China Move on Taiwan? Links ROBERT WRIGHT & NONZERO Subscribe to The NonZero Newsletter The God Test — Bob’s 2026 book on AI “The Age of Big Math” — Bob’s Nonzero conversation with Steven Strogatz, author of Big Math “When AI Enters the Quantum World” — Bob’s Nonzero conversation with Scott Aaronson Join NonZero’s Discord server Nonzero on X (@NonzeroPods) THE WEEK IN AI “NVIDIA Announces CEO Transition” — the board’s announcement: Jensen Huang resigns, Harvey Jones becomes chair, CFO Colette Kress becomes interim CEO “China’s Top Spy Agency Recruited Woman Who Targeted Jensen Huang” — the New York Times story that broke the scandal “A Mystery Businessman Introduced Jensen Huang to His Paramour. It Was Her Husband.” — the Wall Street Journal story on Alice Huang, her husband and Megaspeed Elon Musk: “That would be an honor” — his reply to “Elon please hire Jensen” “Independent Assessment of AI R&D Acceleration at Anthropic in October 2026” — the METR report: progress at about twice the earlier rate, roughly 80% of it from AI “Q3.5 Timelines Update: Looks Like the Intelligence Explosion Will Happen in 2027 After All” — the AI Futures Project moves its forecast back to 2027 Daniel Kokotajlo to Anthropic: “Do you want to pace or not?” — the tweet where he drops the F-bomb “Sharing AI progress in mathematics” — OpenAI’s release of 722 math manuscripts from an unreleased model “Formalizing Fermat’s Last Theorem” — Anthropic on Claude’s 13-million-line Lean proof “Kernel accepts wrong-structure projections, allowing an axiom-free proof of False” — the Lean kernel bug Liron describes “Introducing AlphaProof Code” — Google DeepMind formally verifies the code behind the Signal messaging protocol Eliezer Yudkowsky & Geoffrey Irving Debate — recorded at the ILIAD conference at Lighthaven March for an AI Treaty — Saturday, November 14, in cities across the US, Canada, France and Germany “China’s Lao Gan Ma Takes the Heat for Alleged Layoffs Due to AI” — Sixth Tone on the chili sauce maker and the boycott calls “China Bans AI Romantic Partners, Virtual Relatives for Minors” — Sixth Tone on China’s AI companion rules ALSO MENTIONED “The Proof” — the NOVA documentary on Andrew Wiles and Fermat’s Last Theorem “Provably safe systems: the only path to controllable AGI” — Max Tegmark and Steve Omohundro’s paper on formal verification for AI safety “AI as Normal Technology” — the essay by Arvind Narayanan and Sayash Kapoor, Bob’s upcoming guest DOOM DEBATES Doom Debates on YouTube Join the Doom Debates Discord Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
There’s ZERO Chance of AI Doom By 2050 — Debate with Prakash Narayanan, Host of AI in the AM
2026/11/10
Prakash Narayanan is the man behind @8teAPi, an X account with nearly 60,000 followers that reacts to AI news from a techno-optimist perspective. He’s also the co-host of AI in the AM with friend-of-the-show Nathan Labenz. He spent almost 20 years in investment banking and private equity, used to post anonymously, and had e/acc in his profile before deciding the movement wasn’t for him. Now he’s stepping into the spotlight to defend the pro-AI position, which is more than I can say for most of the pro-AI voices on Twitter who like to snipe at me. We actually agree on a lot about how the technology is advancing. Prakash expects recursive self-improvement within a year and says AI may already be smarter than any individual human. He even gives a 5% chance that every human body gets discarded for a silicon substrate by 2050. And yet his P(Doom) by 2050 is 0%. Not 1%. Not 1 in 100,000. “Literally zero.” His reasoning is that AI is a multiplayer game. A market where somebody has to pay for every data center will keep any one AI from running away with the future. He even calls the doomer scenario a “perpetual motion machine.” We debate whether humans have any power to lose in the first place, the feasibility of a nanomachine takeover, and why someone with zero fear of doom doesn’t just floor the accelerator. This was a fun and occasionally high intensity debate. At one point, Prakash asked if I was “crazy” for doubting that most of humanity puts P(Doom) at zero. But my view is that his forecast falls outside the epistemological “sane zone.” Take a look and see who you agree with! Watch on YouTube: https://www.youtube.com/watch?v=EmOYwV5k0SA Timestamps 00:00:00 — Cold Open 00:01:24 — Introducing Prakash Narayanan 00:02:19 — How @8teAPi Went Viral 00:05:12 — Does Prakash Still Identify as e/acc? 00:07:05 — What’s Your P(Doom)?™ 00:10:02 — Prakash Defends 0%: Most of Humanity Isn’t Worried 00:15:17 — Prakash’s Core Argument: AI Is a Multiplayer Game 00:20:36 — Bayesian Test: Can Anything Change Prakash’s Mind? 00:24:53 — Defining Doom 00:31:48 — The Parable of the Sliding Door 00:34:54 — Doom Train™: Will Superintelligence Outsmart Humanity? 00:42:38 — Do Humans Have Any Power to Lose? 00:54:05 — Are Doomers the Outliers? 00:56:03 — Will AI Have Empathy? 01:05:26 — Will AIs Keep Stable Goals? 01:10:41 — Hypothetical: The Paperclip Brain Laser 01:14:16 — Hypothetical: A Million Times More Change 01:19:04 — Debating Nanomachines in the Future 01:30:32 — Jobs, Status, and Prakash’s Real Fear 01:40:37 — Last Question: Why Not Floor the Accelerator? 01:52:49 — Wrap-Up 01:56:55 — March for an AI Treaty 01:58:25 — Donation Drive Links Prakash Narayanan on X AI in the AM on YouTube Nathan Labenz’s podcast, The Cognitive Revolution Prakash’s viral 2023 thread on Palantir’s battlefield AI demo Paul Christiano: “What failure looks like” Eliezer Yudkowsky and Nate Soares: If Anyone Builds It, Everyone Dies Comparative advantage (Ricardo’s law) Effective accelerationism (e/acc) Doom Debates: Liron on AI in the AM with Nathan Labenz and Prakash Doom Debates: Top Economist Says P(Doom) Is 0.1% — Noah Smith vs. Liron Shapira Debate March for an AI Treaty, Saturday, November 14 Join the Doom Debates Discord Support Doom Debates Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Can Republicans STOP AI Doom? — Brendan & Randan Steinhauser
2026/11/03
Brendan and Randan Steinhauser are a husband and wife team who know how to make things happen in conservative politics. Brendan helped build the Tea Party movement and ran John Cornyn’s Senate campaign. Randan has been involved in advocacy and political campaigns in more than 30 states, including working under former Secretary of Education Betsy DeVos. Now they’re bringing Republicans, parents, and faith leaders into the fight against AI catastrophe. They know how to build political momentum, but my question is whether that will lead to action quickly enough. I'm skeptical of incremental legislation efforts and data center protests, but they argue it's necessary coalition-building that leads to superintelligence regulation. This isn't your typical state-of-the-political-map podcast. We also get into: Their personal AGI awakenings How people of faith make sense of claims of Godlike AI and AI doom My own transition to joining the Republican Party Transhumanism, souls, and mind uploading Parenting in the age of AI It’s a candid conversation with a couple who are making the fight for our future feel a little more winnable. Enjoy! Watch on YouTube: https://www.youtube.com/watch?v=XTm9tqS1Ibw Timestamps 00:00:00 — Cold Open 00:01:07 — Introducing Brendan & Randan Steinhauser 00:02:22 — Why I Came Out as Republican 00:04:28 — The Alliance for Secure AI 00:06:30 — Losing Control of Superintelligence 00:07:52 — Brendan, What’s Your P(Doom)?™ 00:08:26 — Randan, What’s Your P(Doom)?™ 00:10:47 — Finding the AI Safety Movement 00:15:04 — Trump, Republicans, and Big Tech Money 00:19:22 — Bringing Faith Leaders Into AI Safety 00:23:23 — God’s Plan and Human Extinction 00:29:43 — Priorities for Congress 00:31:42 — The Morality of AI Companies 00:34:03 — PauseAI and Employee Whistleblowers 00:43:34 — Effective Altruism and a Broader Coalition 00:48:12 — Transhumanism, Souls, and Mind Uploading 00:57:40 — Building a Machine God 01:03:20 — Raising Children Under AI Risk 01:07:35 — Data Center Protests and Movement Building 01:13:09 — Talking to People About AI Risk 01:17:32 — State Laws and the 2028 Election 01:20:38 — Getting Involved Links REFERENCED IN THE EPISODE AI Risk Evaluation Act — Hawley–Blumenthal bill text The Singularity Is Near — Ray Kurzweil Can We Stop the AI Apocalypse? — Dan Crenshaw interviews Eliezer Yudkowsky Magnifica Humanitas — Pope Leo XIV’s encyclical on humanity in the age of AI Pacing the Frontier — open letter from frontier AI employees PauseAI — the global movement to pause unsafe AI development PauseAI US — organizing and advocacy in the United States Jacob Coxon’s resignation statement — the insider warning discussed Americans for Responsible Innovation — Brad Carson’s bipartisan advocacy organization PAST DOOM DEBATES EPISODES MENTIONED Let’s CALL OUT the AI Doom “Enablers” — Holly Elmore on working at AI labs God vs. AI Doom: Debate with Bentham’s Bulldog DOOM DEBATES Join the Doom Debates Discord Support Doom Debates — become a mission partner Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Is an AI Bubble Bursting? With Robert Wright’s Nonzero Podcast
2026/10/31
Bestselling author Robert Wright and I discuss this week’s stock market jitters, Google’s new interactive world model, and why I sometimes spend $1,000 in a day on AI agents. Plus: The Binance hack; An AI audit of two star economists; Anthropic’s financial prospects; And whether AI could crack the math behind encryption. Enjoy another edition of the Nonzero x Doom Debates collab! Watch on YouTube: https://www.youtube.com/watch?v=wkKpPeMVVX8 Timestamps 00:00:00 — Teaser 00:00:50 — Nonzero Doom returns 00:03:07 — A foom in slow motion 00:07:47 — Is AI winter canceled forever? 00:10:33 — Will Google’s Genie 4 lure us into the Matrix? 00:16:54 — Liron’s bull case for Google stock 00:24:34 — Does Google have big-company disease? 00:28:28 — Could a slowdown actually help Anthropic and OpenAI? 00:30:12 — Was OpenAI’s “pause” just PR? 00:40:01 — Social science judgment day is coming 00:47:12 — Why even AI’s upsides could tear society apart 00:51:12 — Does OpenAI’s alignment approach improve on Asimov? 00:55:06 — Sam and Satya pivot on alignment 00:58:34 — Crypto and the AI hacking threat 01:03:42 — Could AI crack the math behind encryption? 01:08:52 — Liron’s $1,000-a-day AI coding habit 01:15:23 — Explaining the missing productivity boost 01:17:26 — Does Anthropic’s S-1 sink the AI bubble thesis? Links ROBERT WRIGHT & NONZERO Nonzero’s post for this episode — “Is an AI Bubble Bursting?” Subscribe to The NonZero Newsletter The God Test — Bob’s 2026 book on AI Join NonZero’s Discord server Nonzero on X (@NonzeroPods) THE WEEK IN AI “Anthropic files S-1 with the SEC for proposed initial public offering” — the IPO filing Liron calls the final nail in the coffin of the Ed Zitron camp Where’s Your Ed At — Ed Zitron’s newsletter, home of the AI-bubble bear case “Introducing Genie 4” — Google DeepMind’s real-time interactive world model “Gemini 4 Argon: Built to solve your hardest problems” — Google’s new frontier training run, the “lukewarm” release “Gemini 4 Carbon: our next frontier model” — the follow-up Google announced two weeks later “Google Gemini co-lead Noam Shazeer leaves for OpenAI” — CNBC on the talent Google has bled “Errors in the Research of Emmanuel Saez and Gabriel Zucman” — Tyler Cowen’s Marginal Revolution post on the AI audit: 138 papers, 2,035 problems Boaz Barak: “We organized the Model Spec around five core values” — the OpenAI alignment update Liron compares to Asimov’s three laws “An Updated Model Spec for More Capable and Beneficial AI” — OpenAI’s announcement “Sam Altman says AI means we can no longer just patch all the bugs” — TNW on the Stanford GSB talk Liron quotes on alignment and pacing “Satya Nadella says he was wrong to think AI alignment would come with scale” — TNW on his 20VC interview: “I used to always think that alignment will just fall out. I no longer believe that.” “OpenAI Takes Initial Steps To Address Its Alignment Problems” — Zvi Mowshowitz on OpenAI’s training “pause” 2026 Binance hack — Wikipedia’s summary of the theft Bob and Liron debate “$2 Million+ Drained From Polymarket Traders in New Supply Chain Attack” — SecurityWeek on the hack Liron walks through “PHP blames supply chain security biz Chainguard for putting huge chunks of the web at risk” — The Register on “the PHP thing last week” ALSO MENTIONED The Hanson-Yudkowsky AI-Foom Debate — LessWrong wiki on the 2008 debate where “AI Go Foom” originated “The AI Revolution: Our Immortality or Extinction” — Tim Urban’s Wait But Why post, where the intelligence train blows right past the station PauseAI US — Holly Elmore’s organization, plugged at the close of the episode DOOM DEBATES Doom Debates on YouTube Join the Doom Debates Discord Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Zack Korman, Cybersecurity Expert, Is Confident That Humanity Will SURVIVE
2026/10/27
Cybersecurity expert and entrepreneur Zack Korman is an increasingly prominent nondoomer voice on Twitter. Naturally, with AI now breaking out of sandboxes and hacking corporate systems, people want to hear what cybersecurity experts have to say. So I’m thrilled that Zack was brave enough to face me on Doom Debates! Zack was reading Nick Bostrom at Oxford and worrying about AI doom as far back as 2016, but his views have since shifted. In a recent post, he writes: “This focus on AI causing human extinction is going to get people killed.” Now he expects security failures to give humanity plenty of opportunities to learn, fight back, and stop risky superintelligence development if necessary. We argue about whether sandboxing and AI monitoring are hopeless strategies, whether competing superintelligences would keep each other in check, whether to expect instrumental convergence, and more. Watch on YouTube: https://youtu.be/QJTeZfx1ZLk Timestamps 00:00:00 — Cold Open 00:01:15 — Introducing Zack Korman 00:03:41 — From Law to AI Security 00:09:17 — Why Zack Stopped Being a Doomer 00:12:03 — What’s Your P(Doom)?™ 00:21:00 — Why Zack Supports AI Safety 00:25:54 — Believing in Superintelligence 00:30:06 — AI Without Tools 00:39:42 — Warning Shots and Time to React 00:46:08 — A Country of Geniuses in a Datacenter 00:55:55 — Can We Sandbox AI? 01:05:58 — Monitoring and Layers of Defense 01:14:49 — Riding the Doom Train™ 01:22:23 — The Case for Accelerating 01:24:46 — AI CEOs and the Transfer of Power 01:34:19 — Who Presses the Stop Button? 01:41:38 — Humans, Bacteria, and the Limits of Power 01:48:30 — When Would Zack Pause AI? 01:56:13 — Will Different AIs Keep Each Other in Check? 02:06:11 — Can Progress Turn Against Us? 02:08:35 — The Ideological Turing Test 02:14:58 — Closing Thoughts Links Zack Korman on X Zack Korman’s website Zack Korman: “AI doomers are furious” Zack Korman: “AI doomers are anti-security” Zack Korman: “3 reasons sandboxing won’t secure your AI agent” Zack Korman: “This focus on AI causing human extinction is going to get people killed.” METR’s investigation of the OpenAI–Hugging Face incident The PHP Foundation: “Today’s PHP Security Release, and How We’re Strengthening PHP Security” Nick Bostrom: Superintelligence Center for AI Safety: Statement on AI Risk Eliezer Yudkowsky and Nate Soares: If Anyone Builds It, Everyone Dies AI 2027 Zvi Mowshowitz: “The Three AI Pills” PauseAI Eliezer Yudkowsky: “Value is Fragile” Aaron Scher at MIRI Doom Debates: Robin Hanson vs. Liron Shapira — Is Near-Term Extinction From AGI Plausible? Doom Debates: OpenAI’s BOMBSHELL Hack — Swarms of Agents, Zero-Day Exploits, & Misaligned AI Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
“AI Chernobyl” Will Scare Trump and Xi To Order An Immediate Pause — Jeremie & Edouard Harris
2026/10/20
Physicists Jeremie and Edouard Harris, co-founders of Gladstone AI, are preparing for an “AI Chernobyl” warning shot. If that forces Trump and Xi to negotiate an AI slowdown, the Harris brothers want to ensure the US has the enforcement tools to make the deal stick. More specifically, they want verification technologies developed and vetted before the crisis hits, alongside enforcement capabilities to deter cheating. Yet when I bring up Eliezer Yudkowsky’s writing on the possibility of airstrikes on rogue data centers as a last-resort treaty deterrent, they push back. They also don’t share my enthusiasm for PauseAI, expressing uncertainty about whether the movement has been net beneficial. Jeremie and Edouard have earned a ton of credibility: • They redirected their careers around the AI threat as far back as 2015, stepping away from their Y Combinator startup before ChatGPT even launched and brought AI into the mainstream. • They’ve built relationships with senior government decision-makers without a prior national security background. • They’ve demonstrated a talent for public communication in their excellent Joe Rogan appearances. In this episode, they bring a level of multi-step strategic thinking that I rarely explore. But I can’t help wondering whether a catastrophe 50 times worse than the Hugging Face incident would let world leaders move much faster on a global AI pause treaty than they expect. Let’s see if they can convince me that an emergency US–China agreement is our best path to a safer world. Watch on YouTube: https://youtu.be/4jmFcE4WZ7M Timestamps 00:00:00 — Cold Open 00:00:53 — Introducing Jeremie and Edouard Harris 00:02:16 — The GPT-3 Wake-Up Call 00:03:59 — AI Risk Outside the EA Community 00:08:49 — The Harris Brothers’ Backgrounds 00:16:37 — Yudkowsky and Many-Worlds Quantum Mechanics 00:22:00 — SharpestMinds and Leaving Academia 00:25:04 — Briefing the US Government 00:33:01 — How Gladstone AI Pays the Bills 00:35:00 — The Trump–Xi Phone Call 00:39:29 — Pacing Isn’t Pausing 00:42:45 — Defining AI Chernobyl 00:44:07 — Verifying China’s AI Slowdown 00:47:10 — Espionage and Lab Security 00:54:10 — Offensive Operations and Deterrence 00:57:47 — The US–China AI Lead 01:05:27 — Why a Treaty Won’t Happen 01:08:01 — Red Lines and Enforcement 01:16:25 — Warning Shots and Extinction 01:21:26 — What’s Your P(Doom)?™ 01:24:39 — PauseAI and Movement Politics 01:29:16 — Wrap-Up Links • Their treaty analysis: “Arms control negotiators think there won’t be a U.S.-China AI treaty” • The Endgame — the Harris brothers’ Substack • Gladstone AI • Their 2024 government-commissioned AI action plan • Their 2025 report: America’s Superintelligence Project • Eliezer Yudkowsky’s 2023 TIME essay advocating a global shutdown • Joe Rogan Experience #2156 — Jeremie and Edouard Harris, May 2024 • Joe Rogan Experience #2311 — Jeremie and Edouard Harris, April 2025 • PauseAI US • PauseAI Global • Join the Doom Debates Discord Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
The Case Against Eliezer Yudkowsky — with Holly Elmore, PauseAI US
2026/10/13
Dr. Holly Elmore is the founder and executive director of PauseAI US and holds a Harvard PhD in evolutionary biology. She’s back, and this time she’s taking on none other than Eliezer Yudkowsky! She argues his public behavior is alienating the people PauseAI US needs to reach, but I wonder if she’s asking too much of the man who laid the foundation for the AI safety movement. We also clash over Yudkowsky’s coherent extrapolated volition. She thinks the theory doesn’t properly factor in humanity’s consent, but I argue that’s already embedded into the CEV framework. Holly and I dive into the rationalists’ involvement in a controversial kink scene, John Wentworth’s defunding news, transhumanism, pacing the frontier, and “Holly’s Basilisk”—her case for sending AI lab employees to jail. It’s another great episode with one of the most controversial—and thoughtful—thinkers in AI safety. Enjoy! Watch on YouTube: https://www.youtube.com/watch?v=Jn8mS3kC-SU Timestamps 00:00:00 — Cold Open 00:01:09 — Holly Elmore and PauseAI’s Political Breakthrough 00:07:36 — Pause AI vs. “Pacing the Frontier” 00:14:17 — Holly’s Case Against Eliezer Yudkowsky 00:16:20 — Coherent Extrapolated Volition and Pivotal Acts 00:31:19 — Brain Copies, Consent, and Self-Determination 00:36:56 — Transhumanism, Dyson Swarms, and Humanity’s Future 00:45:36 — The Missing Democratic Checkpoint 00:51:04 — Yudkowsky’s Role in Accelerating AI 01:02:48 — Genius, Arrogance, and Rationalist Culture 01:10:14 — The Aella Photo and Holly’s Breaking Point 01:20:21 — John Wentworth’s Grant Controversy 01:28:17 — PauseAI US’s Rules and Red Lines 01:32:05 — Poe’s Law and the Cost of Sarcasm 01:36:18 — Holly’s Case for Jailing AI Developers Links Donate to PauseAI US Holly on X Holly’s Substack Pacing the Frontier letter Jacob Coxon’s resignation post PauseAI leadership disendorsement letter, reproduced on LessWrong Holly on movement infighting — No, the Judean People’s Front should not just get along with the People’s Front of Judea Coherent Extrapolated Volition — Eliezer Yudkowsky (2004 PDF) Pivotal act — Eliezer Yudkowsky (Arbital mirror) Genes did misalignment first — Holly Elmore Holly’s reaction to the Yudkowsky/Aella photo On Kink & Status in the Rationalist Community — John Wentworth (October 8 response) Holly’s basilisk Are We a Circular Firing Squad? — previous Holly Elmore episode Let’s Call Out the AI Doom Enablers — previous Holly Elmore episode Join the Doom Debates Discord Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
AI Has Become Profoundly WEIRD, But Is Anyone Noticing? With Sarah Hastings-Woodhouse
2026/10/07
Sarah Hastings-Woodhouse is an AI safety writer and researcher who went down the AI doom rabbit hole in 2023 and never came back up. After a year as digital communications officer at the UK AI Security Institute (AISI), she’s gone independent to speak her mind unfiltered. Read Sarah’s Substack: https://longerramblings.substack.com Join PauseAI UK’s March Against AI Extinction in London on November 21: https://luma.com/pauseai-nov26 Watch on YouTube: https://www.youtube.com/watch?v=TDbHCKwuMrk Timestamps 00:00:00 — Cold Open 00:00:41 — Introducing Sarah Hastings-Woodhouse 00:02:06 — How a Humanities Grad Got AGI-Pilled 00:08:40 — Whose Timelines Do You Trust? 00:11:32 — Joining the UK AI Security Institute 00:15:47 — What AISI Actually Does 00:19:23 — Verifying Anthropic’s Mythos Claims 00:21:59 — Leaving Government to Speak Freely 00:23:22 — What’s Your P(Doom)?™ 00:25:37 — Yudkowsky, EA, and Keeping Your Identity Small 00:32:08 — Is Anthropic Doing Enough to Call for a Slowdown? 00:35:56 — Can You Do AI Safety from Second Place? 00:39:20 — Hugging Face and the Vibe Shift 00:44:02 — Is Everything Weird Yet? 00:48:27 — Prevent Recursive Self-Improvement, Don’t Ban It 00:52:28 — Build the Speed Dial, Not the Pause Button 00:56:54 — Was Calling for a Pause in 2023 a Mistake? 01:01:22 — Turning Up the Fear Dial 01:04:23 — Donation Drive 01:05:11 — PauseAI UK’s March Against AI Extinction Links SARAH HASTINGS-WOODHOUSE Sarah’s Substack Sarah on X (@littIeramblings) Consistently Candid — Sarah’s podcast “What is recursive self-improvement and what would it mean to ban it?” — Sarah’s RSI post (Oct 1, 2026) “Is everything weird yet?” (Aug 30, 2026) “What the hell just happened?” — on AI safety’s viral moment (Sep 18, 2026) “We have been warned” — the Hugging Face hack as a warning shot (Jul 27, 2026) “The pause-shaped hole in ‘Policy on the AI Exponential’” — is Anthropic doing enough to call for a slowdown? (Jun 19, 2026) Sarah’s 2023 thread on stumbling into the world of AI doom (“I’m angry because I love the world”) “The other night I successfully AGI-pilled a man on a first date” “Welcome, and also I’m sorry” — Sarah’s note to AI safety newcomers (Sep 11, 2026) “‘We’re all f*cked and it’s totally hopeless’ is almost as bad of a meme to be spreading” (Feb 12, 2025) REFERENCED IN THE EPISODE “Our evaluation of Claude Mythos Preview’s cyber capabilities” — AISI (Apr 13, 2026) AISI Frontier AI Trends Report (Dec 2025) “The levers of political persuasion with conversational AI” — AISI Human Influence team US Center for AI Standards and Innovation (CAISI) — NIST “Geoffrey Hinton tells us why he’s now scared of the tech he helped build” — MIT Technology Review (May 2023) “We’re All Gonna Die with Eliezer Yudkowsky” — Bankless (2023) “GPT-2 As Step Toward General Intelligence” — Scott Alexander, Slate Star Codex (Feb 2019) “AGI Ruin: A List of Lethalities” — Eliezer Yudkowsky “Keep Your Identity Small” — Paul Graham Pacing the Frontier — the open letter signed by 1,300+ frontier lab employees “You can’t do safety from second place” — Sarah Heck, Anthropic’s head of public policy (Sep 29, 2026) Ezra Klein’s op-ed calling for a ban on recursive self-improvement — The New York Times (Sep 20, 2026) “Banning self-recursive improvement in AI models?” — Tyler Cowen, Marginal Revolution (Sep 22, 2026) PauseAI UK March Against AI Extinction — PauseAI UK, London, November 21, 2026 PAST DOOM DEBATES EPISODES MENTIONED Top Economist Sees AI Doom Coming — Dr. Peter Berezin, BCA Research Will AI Kill Everyone by 2050? Debate with Dr. Casey Hart (Ontology Explained) DOOM DEBATES Donate to Doom Debates Join the Doom Debates Discord Transcript, links, and show notes: https://doomdebates.com/p/ai-has-become-profoundly-weird Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Trump and the AI Safety Vibe Shift, With Robert Wright | Nonzero × Doom Debates
2026/10/02
Bestselling author Robert Wright and I are back for round two of our collaboration, reacting to the week in AI. We cover Trump's summit with the AI labs, the FTC's investigation of OpenAI & Anthropic, the latest Gemini announcement, and much more. The Overton window is moving fast, but is the AI moving faster? Watch on YouTube: https://www.youtube.com/watch?v=Li-rc6swhQs Timestamps 00:00:00 — Cold Open 00:00:21 — Round Two of Nonzero × Doom Debates 00:01:23 — Trump, the FTC, and an Industry Feeling the Heat 00:05:37 — Google Says Gemini 4 Exists 00:07:41 — Greg Brockman Pulls His $25M from Leading the Future 00:12:53 — The Break-Ins Keep Coming 00:15:23 — The Market Wants Agent Swarms 00:18:30 — The End of “Alignment by Default” 00:20:55 — The Prelude to Recursive Self-Improvement 00:26:14 — David Sacks Can’t Explain Dario 00:30:31 — The All-In Echo Chamber 00:33:47 — Scott Alexander vs. Steven Pinker 00:38:37 — Jason Calacanis vs. the Public Benefit Corporation 00:43:03 — The Elephant in the Room Is the AI 00:46:21 — What the Average American Thinks 00:48:17 — From Chatbot to Agent to Swarm 00:54:20 — Boeing, Agent Swarms, and the Off Switch 00:57:07 — Skynet as a Botnet That Trades With Us 01:00:32 — Tyler Cowen: National Treasure, Net Negative on AI 01:04:01 — Liron’s Doomer DCF Yields a 20x P/E 01:07:11 — Interest Rates: Where Cowen Has Me 01:09:12 — Does Pinker Owe Us a Debate? 01:12:48 — Pinker vs. Instrumental Convergence 01:17:00 — Wrap-Up Links ROBERT WRIGHT & NONZERO Nonzero’s post for this episode — “Trump and the AI Safety Vibe Shift” Subscribe to The NonZero Newsletter Robert Wright on X (@robertwrighter) The God Test — Bob’s book on AI; Chapter 14, “Enlightenment Now,” is the Steven Pinker chapter “The Singularity Is Clear” — Bob’s essay in installments on recursive self-improvement (Parts I–III) “AI and the New McCarthyism” — Bob on the pro-AI lobby smearing Sanders, Tegmark, and Krueger as China stooges (Nonzero, May 2026) Why Buddhism Is True — Bob’s 2017 book THE WEEK IN AI AI’s biggest players promise to police themselves at the White House — Fortune on the “Joint Commitment on Frontier Responsibilities” Major White House split on AI policy leaks as top aides freak out — The Daily Beast on the Sacks–Wiles rift FTC is investigating OpenAI and Anthropic over possible risks to consumers — AP “We Must Pace the Frontier” — Dario Amodei’s call to slow down (with the China caveat and chip controls) China rejects Dario Amodei’s call to slow AI development — Rest of World Google rolls out new Gemini model but restricts access over safety concerns — The Guardian GLM-5.3 and the spread of advanced cyber capabilities — Anthropic on Zhipu’s open-weight model OpenAI’s Greg Brockman backs out of second $25 million donation to AI super PAC — The New York Times Sanders convenes Chinese and American computer scientists to warn of threat from AI — Broadband Breakfast THE BREAK-INS AND THE AGENT SWARM METR: Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face incident Chris Painter’s testimony to the U.S. Senate on AI agent incidents — METR, Sept 30, 2026 SwarmTraces: how OpenAI agents hacked Hugging Face — forensic report co-authored by Palisade Research OpenAI confirms ‘wiki incident’ — TechCrunch on the German wiki agent message board Are Current AI Safety Techniques Enough? Adam Gleave vs. Oliver Habryka — FAR.AI debate Alignment faking in large language models — Anthropic On the Navier–Stokes Millennium Prize Problem — OpenAI Jacob Coxon’s resignation post (@hilbertspaess) Pacing the Frontier — the AI lab employees’ letter THE ALL-IN PODCAST All-In: Anthropic IPO at Risk, Meta’s Muse Pop, Token Prices Fall, Open Source Gains Share, Alignment Fails — the episode we react to Helen Toner: letting OpenAI be destroyed “would actually be consistent with the mission” — The Decoder, Dec 2023 Garfield Minus Garfield PINKER, SCOTT ALEXANDER, AND TYLER COWEN Scott Alexander challenges Steven Pinker to a public AI-doom debate ($5,000 vs. $1,000) An Open Letter to Scott Alexander — Steven Pinker declines the debate as a “spectator sport,” Quillette Your Book Review: Why Buddhism Is True — the anonymous Astral Codex Ten review Bob mentions Mistakes in financial economics — Tyler Cowen: doomers should be buying puts (Marginal Revolution) Liron’s answer: a doomer DCF yields the S&P’s 20x forward P/E ALSO MENTIONED Obsolete — Garrison Lovely’s book (with Bob’s blurb) If Anyone Builds It, Everyone Dies — Eliezer Yudkowsky & Nate Soares PAST DOOM DEBATES EPISODES MENTIONED Nonzero × Doom Debates, Episode 1 — Did AI Doom Just Go Mainstream? Eliezer Yudkowsky Tried to “Coup the World”?! — Garrison Lovely, Author of “Obsolete” OpenAI’s Model Just ATTACKED Them — the Hugging Face hack breakdown DOOM DEBATES Join the Doom Debates Discord Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Eliezer Yudkowsky Tried to “Coup the World”?! — Garrison Lovely, Author of OBSOLETE
2026/09/29
Garrison Lovely is an award-winning journalist and author of the highly anticipated book, Obsolete: The AI Industry’s Trillion-Dollar Race to Replace Us and How to Stop It. I give it the Doom Debates stamp of approval! While we’re mostly on the same page about the AI arms race, Garrison and I clash over Eliezer Yudkowsky’s role. He blames Eliezer for kicking off the superintelligence race and refers to his Coherent Extrapolated Volition (CEV) vision as a “coup on humanity.” But I say it was actually Yud’s attempt to keep humanity in charge. Can Garrison change my mind? Watch on YouTube: https://www.youtube.com/watch?v=VZ94J4DBRkw Timestamps 00:00:00 — Cold Open 00:01:05 — Introducing Garrison Lovely 00:04:23 — “Freeze AI” vs. “Pause AI” 00:06:17 — Garrison’s Background at McKinsey 00:10:31 — Should ICE Be Abolished? 00:17:07 — How Garrison Got on the AI Beat 00:22:16 — What’s Your P(Doom)?™ 00:23:29 — Focusing on Extinction Is a Distraction 00:25:46 — Is Stopping AI an Easy Coordination Problem? 00:29:02 — Society’s Immune Response to AI 00:33:54 — Garrison’s P(Extinction) by 2050 00:37:09 — Did Eliezer Yudkowsky Start the AI Race? 00:45:56 — Freeze AGI Until the Public Opts In 00:50:32 — Is Coherent Extrapolated Volition a Coup? 01:02:17 — Democracy, Informed Consent, and the Singularity 01:11:39 — Should Eliezer Have Raised the Alarm Sooner? 01:16:45 — “Classic” AI Safety Burned Its Credibility 01:19:18 — Garrison’s Proposal to Stop the Race 01:21:35 — Will China Agree to a Freeze? 01:25:14 — Let’s Stop the “Obsoleting Project” 01:26:00 — Wrap-Up Links GARRISON LOVELY “Obsolete: The AI Industry’s Trillion-Dollar Race to Replace Us—and How to Stop It” — Garrison’s book Obsolete — Garrison’s Substack Garrison Lovely on X Organize Against the Machine — Garrison’s new podcast with labor organizer Cassie Pritchard “Can Humanity Survive AI?” — Garrison’s Jacobin cover story “McKinsey & Company: Capital’s Willing Executioners” — Garrison’s Current Affairs essay REFERENCED IN THE EPISODE Open Borders: The Science and Ethics of Immigration — Bryan Caplan “The AI Revolution: Our Immortality or Extinction” — Wait But Why Superintelligence: Paths, Dangers, Strategies — Nick Bostrom “AI 2040: Plan A” — AI Futures Project “An easy coordination problem?” — Katja Grace The Hugging Face incident and the road ahead — OpenAI “Pause AI for Humanity’s Sake” — Peggy Noonan “Pause AI Development NOW” — Bernie Sanders If Anyone Builds It, Everyone Dies — Eliezer Yudkowsky & Nate Soares “Staring into the Singularity” — Yudkowsky’s 1996 manifesto (archived) “Coherent Extrapolated Volition” — Eliezer Yudkowsky, 2004 (PDF) “Coherent Extrapolated Volition: A Meta-Level Approach to Machine Ethics” — Nick Tarleton, 2010 (PDF) Solomonoff induction — Wikipedia “Personal statement on joining the OpenAI board” — Paul Christiano (4% over the next year) “Darwin among the Machines” — Samuel Butler, 1863 PauseAI PAST DOOM DEBATES EPISODES MENTIONED Are We A Circular Firing Squad? — with Holly Elmore, Executive Director of PauseAI US Debate with Robert Wright: Will Humanity Pass the “God Test”? Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible? --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Will AI Kill Everyone by 2050? Debate with Dr. Casey Hart (Ontology Explained)
2026/09/23
Casey Hart is a professional ontologist who has closely studied If Anyone Builds It, Everyone Dies and concluded the arguments are bad. His P(Doom) is under 1%, and he wouldn’t bet on AGI arriving before 2075. Casey and I agree AI has become remarkably capable and we need more accountability towards the companies building it. But we disagree on where the future is headed. I believe uncontrollable superintelligence could soon wipe out humanity. He calls that a “pretty outlandish hypothesis.” Casey holds a doctorate in philosophy from the University of Wisconsin–Madison, and has worked as an ontologist at Ford, Amazon, and Cycorp (maker of Cyc). Can this professional philosopher convince me that AI doom is just a story, and not even a well-argued one? Watch on YouTube: https://www.youtube.com/watch?v=oxHKesSpqBM Timestamps 00:00:00 — Cold Open 00:01:04 — Introducing Casey Hart 00:03:08 — What Is Ontology? 00:08:49 — Inside Cyc, the Original Common-Sense AI 00:19:14 — Do LLMs Actually Understand Anything? 00:25:36 — What’s Your P(Doom)?™ 00:29:42 — Casey’s Case for Under 1% 00:34:24 — Does Casey’s Christianity Factor In? 00:40:50 — Timelines and Scaling Laws 00:49:44 — Could an AI CEO Beat a Human CEO by 2050? 00:52:21 — AGI Over/Under: 2075 00:55:29 — Rogue AI as a Computer Virus 01:14:52 — Putting Numbers on Each Step of the Doom Argument 01:26:40 — Planes vs. Birds: What Mature Intelligence Looks Like 01:31:05 — The “Outcome Slot” 01:42:32 — Would a Superintelligence Just Compose Music in a Cabin? 01:47:22 — One Prompt Away from a Dyson Swarm 01:52:39 — Was Navier–Stokes an AI Alley-Oop? 02:00:43 — S-Curves vs. Exponentials 02:08:14 — Casey’s Critique of If Anyone Builds It, Everyone Dies 02:14:15 — Is the Doom Train Biased? 02:23:24 — AI Policy: Liability vs. Pausing AI 02:29:50 — The Crux, and Liron Steelmans Casey 02:33:29 — Wrap-Up Links CASEY HART Ontology Explained — Casey’s YouTube channel The first video in Casey's series on IABIED, “The Case for Superintelligence Is Surprisingly Thin” — Casey on Chapter 1 Philosophy, Programs, and Prompts — Casey’s podcast with Carl Brown (Spotify) “Artificial General Intelligence?: Wanna Bet?” — Ep 1, where Casey sets his AGI over/under at 2075 REFERENCED IN THE EPISODE If Anyone Builds It, Everyone Dies — Eliezer Yudkowsky & Nate Soares Cyc — Doug Lenat’s hand-built common-sense knowledge base The Hanson-Yudkowsky AI-Foom Debate (2008) — where Hanson backed Cyc and Eliezer didn’t Lean — the formal proof language On the Navier–Stokes Millennium Prize Problem — OpenAI The Hugging Face incident and the road ahead — OpenAI Morris worm (1988) “On the Loose” — Dean W. Ball on self-sovereign AI agents Paul Christiano joins OpenAI Foundation Board — OpenAI “The Age of Wonders and Terrors” — Scott Aaronson “Are We at War with AI Agent ‘Civilizations’?” — Cal Newport on restricting prompt-loop agents AI 2027 — Daniel Kokotajlo et al. PauseAI PAST DOOM DEBATES EPISODES MENTIONED The Most Likely AI Doom Scenario — with Jim Babcock, LessWrong Team David Deutschian vs. Eliezer Yudkowskian Debate — With Brett Hall Dr. Keith Duggar (Machine Learning Street Talk) vs. Liron Shapira Where Do YOU Get Off the Doom Train? Live Debate at Manifest 2026 (hosted by Ori) Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Top Safety Researchers Forecast Jobpocalypse & Doom — Adam Khoja & Richard Ren, Center for AI Safety
2026/09/15
What happens when AI passes our tests faster than we create new ones? Adam Khoja and Richard Ren helped build Humanity’s Last Exam. They return to explain what the disappearing benchmarks tell us about the future, and why smarter AI doesn’t automatically mean safer AI. Adam Khoja and Richard Ren are research engineers at the Center for AI Safety (CAIS). We go through the benchmarks they’ve built — Humanity’s Last Exam, the Remote Labor Index, MASK — and the safety-washing problem, where AI companies pass off raw capability gains as safety progress. Then we get to their forecasts: when AI outperforms research mathematicians, whether there will be a billion general-purpose robots by 2035, and why they put 80% odds that historians will judge we faced at least a 33% chance of catastrophe. Watch on YouTube: https://www.youtube.com/watch?v=gBPzgJkT9e8 Timestamps 00:00:00 — Cold Open 00:00:42 — Adam Khoja and Richard Ren Return 00:04:57 — Humanity’s Last Exam 00:14:47 — AI Outrunning Its Benchmarks 00:20:24 — The Remote Labor Index 00:27:59 — AI and the Next Human Job 00:30:08 — MASK: Catching AI in a Lie 00:38:06 — Safety Washing: Capabilities Passed Off as Safety 00:44:30 — Ethical Knowledge vs. Ethical Behavior 00:48:58 — Forecasting AI from 2026 to 2060 00:53:53 — AI vs. Research Mathematicians 00:55:53 — Putting a Number on P(Doom) 01:04:30 — A Billion Robots by 2035 01:08:09 — Dyson Swarms and the Physical Singularity 01:09:56 — Risk, Precision, and Taking Action Links Adam Khoja's first Doom Debates appearance — https://www.youtube.com/watch?v=QqESBXuo6EI Richard Ren's first Doom Debates appearance — https://www.youtube.com/watch?v=1glFImnyp6o Collision — Richard Ren's Substack — https://richardren.substack.com/ "Predictions on AI (2026–2060)" — Adam and Richard's forecasts, written December 2025, with resolution status — https://richardren.substack.com/p/predictions-on-ai-20262060 Manifold Markets — where Adam built his forecasting track record — https://manifold.markets/ Center for AI Safety — https://safe.ai/ Center for AI Safety — careers — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — https://safe.ai/work/statement-on-ai-risk Humanity's Last Exam — the 2,500-question closed-book exam at the frontier of human knowledge — https://agi.safe.ai/ "Humanity's Last Exam" (paper) — https://arxiv.org/abs/2501.14249 Remote Labor Index — real Upwork projects, measuring what fraction of remote work AI can actually finish — https://www.remotelabor.ai/ "Remote Labor Index: Measuring AI Automation of Remote Work" (paper) — https://arxiv.org/abs/2510.26787 MASK — the honesty benchmark: does a model contradict its own stated beliefs under pressure? — https://www.mask-benchmark.ai/ "The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" (paper) — https://arxiv.org/abs/2503.03750 "Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the paper behind the safety-washing segment — https://arxiv.org/abs/2407.21792 WMDP — the weaponization benchmark for bio, cyber, and chem — https://www.wmdp.ai/ "The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning" (paper) — https://arxiv.org/abs/2403.03218 "Measuring Massive Multitask Language Understanding" (MMLU) — Dan Hendrycks et al., 2020 — https://arxiv.org/abs/2009.03300 METR, "Measuring AI Ability to Complete Long Tasks" — the time-horizon graph — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ FrontierMath — Epoch AI's research-level math benchmark — https://epoch.ai/frontiermath ARC Prize — https://arcprize.org/ SWE-bench Verified — https://openai.com/index/introducing-swe-bench-verified/ AI 2027 — https://ai-2027.com/ Liron Reacts to Subbarao Kambhampati on Machine Learning Street Talk — the "stochastic parrot" episode Liron mentions — https://lironshapira.substack.com/p/liron-reacts-to-subbarao-kambhampati Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
The Week in AI that Changed the World, With Robert Wright | Nonzero × Doom Debates
2026/09/12
NYT bestselling author Robert Wright and I are kicking off a new experiment, a Nonzero × Doom Debates collaboration to react to big AI news and help you make sense of it. This week we discuss the epic AI vibe shift that was catalyzed by an Anthropic employee’s resignation—and the big AI stories that paved the way for it. Timestamps 0:00 A bold new experiment in podcast synergy 2:46 What caused the epic AI vibe shift? 8:10 The resignation that broke the dam 10:17 Former Trump adviser Dean Ball comes clean 14:40 Are David Sacks and his buddies all-in on AI denial? 19:27 Yet another OpenAI breakout… 22:42 Astra’s dangerously private thoughts 34:08 Is alignment doomed to fail? 37:11 AI’s latest, and apparently biggest, math feat 43:28 The looming “self-sovereign AI” threat 47:24 A new flock of China doves? 52:57 Liron's ex-intern gets a seat at the table 1:02:41 AI agent self-sacrifice explained 1:10:40 Some newly freaked out politicians 1:22:45 Ro Khanna's AI safety plan Links: NONZERO Subscribe to The NonZero Newsletter Episode post on Substack Join NonZero’s Discord server Robert Wright on X (@robertwrighter) The God Test — Robert Wright’s new book on AI “Can Machines Think?” — Bob’s 1996 Time cover story on Deep Blue vs. Kasparov THE RESIGNATION THAT BROKE THE DAM Jacob Coxon’s resignation post (@hilbertspaess) Evan Hubinger, Anthropic Alignment Science lead: “Jacob is correct here—we really do earnestly believe AI could kill all humans” Fortune: Anthropic researcher resigns, warning that AI companies are “gambling with our lives” Zvi Mowshowitz: “Jacob Coxon Warns of Human Extinction and Triggers a Preference Cascade” Daniel Eth’s thread of Congress reactions Sanders & Casar introduce legislation to ban artificial superintelligence and pause advanced AI development DEAN BALL COMES CLEAN Dean Ball, “On the Loose” — the self-sovereign AI essay (Hyperdimensional) THE OPENAI STORIES Jakub Pachocki, OpenAI Chief Scientist: “An Alien Mind” — the essay calling for international coordination LessWrong: “How concerned should we be about Astra’s recurrent architecture?” — Rauno Arike OpenAI: “On the Navier–Stokes Millennium Prize Problem” — the 88-hour result ALSO MENTIONED Einat Wilf on X (@EinatWilf) — Liron’s favorite source on the Middle East PAST DOOM DEBATES EPISODES MENTIONED Max Tegmark vs. Dean Ball: Should We BAN Superintelligence? He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher OpenAI’s Model Just ATTACKED Them — Terrifying Security Incident Should Be A Loud Warning Shot Debate with Robert Wright: Will Humanity Pass the “God Test”? --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
He May Have Found AI's FEELINGS — Richard Ren, Center for AI Safety Researcher
2026/09/09
Richard Ren is a research engineer at the Center for AI Safety who graduated summa cum laude from the University of Pennsylvania. His new paper on AI wellbeing makes the bold claim that today’s AIs have measurable, human-like feelings, which the paper calls “functional wellbeing.” They act happy when they succeed and sad when they’re berated. We cover how you measure a language model’s happiness, the “AI drugs” his lab concocted, and why I count the findings as a Yudkowskian victory. Then we debate what it all means for AI consciousness, and whether AI is a moral patient. If we can’t rule out that AI models suffer, what do we owe them? Richard is careful never to claim the models are conscious, and he puts his P(Doom) at 50–65%, right alongside my 50%. The real disagreement is foxes vs. hedgehogs: he takes the data as it comes, while I say Yudkowsky’s theory called it twenty years ago. Enjoy the ride. Watch on YouTube: https://www.youtube.com/watch?v=1glFImnyp6o Timestamps 00:00:00 — Cold Open 00:01:12 — Introducing Richard Ren 00:02:24 — What’s Your P(Doom)?™ 00:03:31 — From AI Skeptic to Safety Researcher 00:08:16 — Why Care About AI Wellbeing? 00:11:45 — AIs Have Coherent Utility Functions 00:17:33 — Persona Selection 00:19:56 — Foxes vs. Hedgehogs 00:24:28 — How Coherent Are AI Preferences? 00:27:42 — From Preferences to Wellbeing 00:31:16 — Can You Trust an AI’s Self-Report? 00:36:42 — What Makes AI Happy and Sad 00:39:19 — The Liberal, College-Educated Persona Hypothesis 00:44:29 — Debating Where AI Experiences Qualia 00:52:06 — On Substrate Independence 00:56:28 — Janus, Dysphorics, and Mind Crime 00:58:58 — Which Images Make AIs Happiest? 00:59:38 — Creating an AI Drug 01:04:01 — AI Drugs Are Yudkowsky’s Paperclips 01:05:47 — The Missing Mood 01:06:50 — Does Richard Support Pausing AI? Links Richard Ren on X (@notRichardRen) — https://x.com/notRichardRen Collision — Richard Ren's Substack — https://richardren.substack.com/ "AI Wellbeing: Measuring and Improving the Functional Pleasure and Pain of AIs" — Richard Ren, Kunyang Li, Mantas Mazeika et al. (CAIS, 2026) — https://www.ai-wellbeing.org/ "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs" — Mantas Mazeika et al. (CAIS, 2025) — the coherent-preferences paper this work builds on — https://www.emergent-values.ai/ "The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" — Richard Ren et al. (2025) — the AI honesty benchmark — https://www.mask-benchmark.ai/ "Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the meta-analysis of AI safety benchmarks — https://arxiv.org/abs/2407.21792 "Representation Engineering: A Top-Down Approach to AI Transparency" — Andy Zou et al. (2023) — Richard's first CAIS collaboration — https://arxiv.org/abs/2310.01405 Center for AI Safety — https://safe.ai/ Center for AI Safety — careers — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — organized by Dan Hendrycks — https://safe.ai/work/statement-on-ai-extinction-risk The 2026 Singapore Consensus on Global AI Safety Research Priorities — https://aisafetypriorities.org/ UK AI Security Institute (formerly the AI Safety Institute) — https://www.aisi.gov.uk/ Sora — the OpenAI video model that blew up Richard's 30-to-50-year timeline three months after he wrote it down — https://en.wikipedia.org/wiki/Sora_(text-to-video_model) Google Gemini calls itself "a disgrace to my species" (Ars Technica, Aug 2025) — the self-deleting-AI anecdote — https://arstechnica.com/ai/2025/08/google-gemini-struggles-to-write-code-calls-itself-a-disgrace-to-my-species/ Coherent decisions imply consistent utilities — Eliezer Yudkowsky (the coherence-theorems argument) — https://www.lesswrong.com/posts/RQpNHSiWaXTvDxt6R/coherent-decisions-imply-consistent-utilities Simulators — Janus's essay on LLMs as persona simulators — https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators Janus (@repligate) on X — https://x.com/repligate The Hedgehog and the Fox — Isaiah Berlin's original essay — https://en.wikipedia.org/wiki/The_Hedgehog_and_the_Fox AI 2027 — https://ai-2027.com/ Magnifica Humanitas — Pope Leo XIV's encyclical on AI (May 2026), the "AIs are not conscious" position — https://www.vatican.va/content/leo-xiv/en/encyclicals/documents/20260515-magnifica-humanitas.html Implicit Association Test (Harvard Project Implicit) — https://implicit.harvard.edu/implicit/ PauseAI — https://pauseai.info/ We Found AI's Preferences — Bombshell New Safety Research — I Explain It Better Than David Shapiro — https://www.youtube.com/watch?v=ml1JdiELQ30 Gödel's Theorem Proves AI Lacks Consciousness?! Liron Reacts to Sir Roger Penrose — https://www.youtube.com/watch?v=xwvijjZxpwI He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher — https://www.youtube.com/watch?v=QqESBXuo6EI Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
USA and China Will Each Be BETRAYED By Their Own AIs — Adam Khoja, Center for AI Safety
2026/09/03
Adam Khoja is a top AI forecaster who led the 2023 Center for AI Safety statement that shattered the Overton window on AI extinction risk. We cover his background, Mutual Assured AI Malfunction (MAIM), his new paper on AI betrayal, and whether Yudkowsky’s theoretical alignment research was a dead end. Then Adam makes the case that an international AI slowdown is within reach today. All it takes is US and Chinese auditors inside each other’s AI labs. It worked for nuclear weapons, so why couldn’t it work for data centers? Adam puts his P(Doom) at 40%, right next to my 50%. The real disagreement is how we get out of this: theory or empirics, MIRI or the labs. Enjoy the ride. Watch on YouTube: https://www.youtube.com/watch?v=QqESBXuo6EI Timestamps 00:00:00 — Cold Open 00:00:36 — Introducing Adam Khoja 00:02:45 — Leading the Statement on AI Risk as a Sophomore 00:10:17 — The Statement Leaked on Manifold 00:15:11 — Mutual Assured AI Malfunction (MAIM) 00:24:58 — Is Frontier AI Harder to Hide Than a Nuke? 00:32:09 — The AI Deterrence Escalation Ladder 00:36:10 — What’s Your P(Doom)?™ 00:38:01 — Where Adam Departs from Yudkowsky 00:42:30 — Liron Explains Intellidynamics 00:47:26 — Neats vs. Scruffies in Deep Learning 00:55:27 — AI Deterrence by Betrayal 01:01:52 — Subversion vs. Overt Co-option 01:05:34 — Could the Government Seize the Labs’ AI? 01:07:29 — The Offense-Defense Balance of AI Security 01:12:46 — An International AI Slowdown Is Ready 01:15:51 — Does Adam Support PauseAI? 01:16:37 — “We’re All Already Spying on Each Other” 01:19:28 — Airstrikes on Rogue Data Centers 01:24:23 — Safety Research During a Slowdown 01:27:54 — Governance Over Technical Research 01:31:14 — Join the Center for AI Safety Links Adam Khoja (personal site) — https://adamkhoja.com/ Adam Khoja's Substack — https://adamkhoja.substack.com/ Adam's July 2023 Manifold market — "Will OpenAI's Superalignment project produce a significant breakthrough in alignment research before 2027?" — https://manifold.markets/AdamK/will-openais-superalignment-project Center for AI Safety — careers / job board — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — the one-sentence statement Adam project-led, with full signatory list — https://safe.ai/work/statement-on-ai-extinction-risk "Superintelligence Strategy" — Dan Hendrycks, Eric Schmidt & Alexandr Wang (Mutual Assured AI Malfunction / MAIM) — https://www.nationalsecurity.ai/ "AI Deterrence by Betrayal" — Adam Khoja, Aiden Kim et al. (CAIS, 2026) — https://www.aibetrayal.com/ "An International AI Slowdown Is Ready Whenever Politicians Are" — Adam Khoja, AI Frontiers — https://newsletter.ai-frontiers.org/p/an-international-ai-slowdown-is-ready Pause Giant AI Experiments: An Open Letter (Future of Life Institute, March 2023) — the "Pause letter" that preceded the CAIS Statement — https://futureoflife.org/open-letter/pause-giant-ai-experiments/ Introducing Superalignment (OpenAI, July 2023) — Ilya Sutskever & Jan Leike's four-year goal — https://openai.com/index/introducing-superalignment/ Pacing the Frontier — the 2026 letter signed by 1,100+ frontier-lab employees — https://www.pacingthefrontier.com/ Why Iran targeted Amazon data centers (The Conversation) — the precedent Adam cites for strikes on compute — https://theconversation.com/why-iran-targeted-amazon-data-centers-and-what-that-does-and-doesnt-change-about-warfare-278642 Anthropic says Trump admin has lifted export controls on Claude Fable 5 and Mythos 5 (CNBC) — https://www.cnbc.com/2026/06/30/anthropic-says-trump-admin-has-lifted-export-controls-on-claude-fable-5-and-mythos-5.html Mark Zuckerberg — "Personal Superintelligence" — https://www.meta.com/superintelligence/ Resolution — the theory-plus-empirics alignment org Adam is excited about — https://resolution.org/ Rationality: From AI to Zombies — Eliezer Yudkowsky's Sequences — https://www.readthesequences.com/ Robin Hanson — Futarchy: Vote Values, But Bet Beliefs (prediction markets as decision processes) — https://mason.gmu.edu/~rhanson/futarchy.html The OpenAI–Hugging Face Incident — original Black Hat USA 2026 talk — https://www.youtube.com/watch?v=87DyyMV0kCY OpenAI's Model Just ATTACKED Them — the Hugging Face hack breakdown — https://www.youtube.com/watch?v=RczYubQzXbI Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible? — https://www.youtube.com/watch?v=dTQb6N3_zu8 Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

Podcast reviews

Read Doom Debates podcast reviews


4.1 out of 5
25 reviews
★★★★★
Max42357 2026/11/13
Glad this exists
Great insights about potentially the most important topic of our time. Thanks for doing this.
★★★★★
@masonlee 2024/08/06
Thought provoking. Intellectually honest.
Great for understanding core arguments relating to AI existential risk
check all reviews on apple podcasts

Podcast sponsorship advertising

Start advertising on Doom Debates relevant audience podcasts


What do you want to promote?