Advertise on podcast: LessWrong (Curated & Popular)

When economists think and write about the post-AGI world, they often rely on the implicit assumption that parameters may change, but fundamentally, structurally, not much happens. And if it does, it's maybe one or two empirical facts, but nothing too fundamental. This mostly worked for all sorts of other technologies, where technologists would predict society to be radically transformed e.g. by everyone having most of humanity's knowledge available for free all the time, or everyone having an ability to instantly communicate with almost anyone else. [1] But it will not work for AGI, and as a result, most of the econ modelling of the post-AGI world is irrelevant or actively misleading [2], making people who rely on it more confused than if they just thought “this is hard to think about so I don’t know”. Econ reasoning from high level perspective Econ reasoning is trying to do something like projecting the extremely high dimensional reality into something like 10 real numbers and a few differential equations. All the hard cognitive work is in the projection. Solving a bunch of differential equations impresses the general audience, and historically may have worked as some sort of proof of [...] --- Outline: (00:57) Econ reasoning from high level perspective (02:51) Econ reasoning applied to post-AGI situations The original text contained 10 footnotes which were omitted from this narration. --- First published: February 4th, 2026 Source: https://www.lesswrong.com/posts/fL7g3fuMQLssbHd6Y/post-agi-economics-as-if-nothing-ever-happens --- Narrated by TYPE III AUDIO.

The recent book “If Anyone Builds It Everyone Dies” (September 2025) by Eliezer Yudkowsky and Nate Soares argues that creating superintelligent AI in the near future would almost certainly cause human extinction: If any company or group, anywhere on the planet, builds an artificial superintelligence using anything remotely like current techniques, based on anything remotely like the present understanding of AI, then everyone, everywhere on Earth, will die. The goal of this post is to summarize and evaluate the book's key arguments and the main counterarguments critics have made against them. Although several other book reviews have already been written I found many of them unsatisfying because a lot of them are written by journalists who have the goal of writing an entertaining piece and only lightly cover the core arguments, or don’t seem understand them properly, and instead resort to weak arguments like straw-manning, ad hominem attacks or criticizing the style of the book. So my goal is to write a book review that has the following properties: Written by someone who has read a substantial amount of AI alignment and LessWrong content and won’t make AI alignment beginner mistakes or misunderstandings (e.g. not knowing about the [...] --- Outline: (07:43) Background arguments to the key claim (09:21) The key claim: ASI alignment is extremely difficult to solve (12:52) 1. Human values are a very specific, fragile, and tiny space of all possible goals (15:25) 2. Current methods used to train goals into AIs are imprecise and unreliable (16:42) The inner alignment problem (17:25) Inner alignment introduction (19:03) Inner misalignment evolution analogy (21:03) Real examples of inner misalignment (22:23) Inner misalignment explanation (25:05) ASI misalignment example (27:40) 3. The ASI alignment problem is hard because it has the properties of hard engineering challenges (28:10) Space probes (29:09) Nuclear reactors (30:18) Computer security (30:35) Counterarguments to the book (30:46) Arguments that the books arguments are unfalsifiable (33:19) Arguments against the evolution analogy (37:38) Arguments against counting arguments (40:16) Arguments based on the aligned behavior of modern LLMs (43:16) Arguments against engineering analogies to AI alignment (45:05) Three counterarguments to the books three core arguments (46:43) Conclusion (49:23) Appendix --- First published: January 24th, 2026 Source: https://www.lesswrong.com/posts/qFzWTTxW37mqnE6CA/iabied-book-review-core-arguments-and-counterarguments --- Narrated by TYPE III AUDIO. --- Images from the article:

Author's note: this is somewhat more rushed than ideal, but I think getting this out sooner is pretty important. Ideally, it would be a bit less snarky. Anthropic[1] recently published a new piece of research: The Hot Mess of AI: How Does Misalignment Scale with Model Intelligence and Task Complexity? (arXiv, Twitter thread). I have some complaints about both the paper and the accompanying blog post. tl;dr The paper's abstract says that "in several settings, larger, more capable models are more incoherent than smaller models", but in most settings they are more coherent. This emphasis is even more exaggerated in the blog post and Twitter thread. I think this is pretty misleading. The paper's technical definition of "incoherence" is uninteresting[2] and the framing of the paper, blog post, and Twitter thread equivocate with the more normal English-language definition of the term, which is extremely misleading. Section 5 of the paper (and to a larger extent the blog post and Twitter) attempt to draw conclusions about future alignment difficulties that are unjustified by the experiment results, and would be unjustified even if the experiment results pointed in the other direction. The blog post is substantially LLM-written. I think this [...] --- Outline: (00:39) tl;dr (01:42) Paper (06:25) Blog The original text contained 3 footnotes which were omitted from this narration. --- First published: February 4th, 2026 Source: https://www.lesswrong.com/posts/ceEgAEXcL7cC2Ddiy/anthropic-s-hot-mess-paper-overstates-its-case-and-the-blog --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

tl;dr: You can pledge to join a big protest to ban AGI research at ifanyonebuildsit.com/march, which only triggers if 100,000 people sign up. The If Anyone Builds It website includes a March page, wherein you can pledge to march in Washington DC, demanding an international treaty to stop AGI research if 100,000 people in total also pledge. I designed the March page (although am not otherwise involved with March decisionmaking), and want to pitch people on signing up for the "March Kickstarter." It's not obvious that small protests do anything, or are worth the effort. But, I think 100,000 people marching in DC would be quite valuable because it showcases "AI x-risk is not a fringe concern. If you speak out about it, you are not being a lonely dissident, you are representing a substantial mass of people." The current version of the March page is designed around the principle that "conditional kickstarters are cheap." MIRI might later decide to push hard on the March, and maybe then someone will bid for people to come who are on the fence. For now, I mostly wanted to say: if you're the sort of person who would fairly obviously come to [...] --- Outline: (01:54) Probably expect a design/slogan reroll (03:10) FAQ (03:13) Whats the goal of the Dont Build It march? (03:24) Why? (03:55) Why do you think that? (04:22) Why does the pledge only take effect if 100,000 people pledge to march? (04:56) What do you mean by international treaty? (06:00) How much notice will there be for the actual march? (06:14) What if I dont want to commit to marching in D.C. yet? --- First published: February 2nd, 2026 Source: https://www.lesswrong.com/posts/HnwDWxRPzRrBfJSBD/conditional-kickstarter-for-the-don-t-build-it-march --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A low-effort guide I dashed off in less than an hour, because I got riled up. Try not to hire a team. Try pretty hard at this. Try to find a more efficient way to solve your problem that requires less labor – a smaller-footprint solution. Try to hire contractors to do specific parts that they’re really good at, and who have a well-defined interface. Your relationship to these contractors will mostly be transactional and temporary. If you must, try hiring just one person, a very smart, capable, and trustworthy generalist, who finds and supports the contractors, so all you have to do is manage the problem-and-solution part of the interface with the contractors. You will need to spend quite a bit of time making sure this lieutenant understands what you’re doing and why, so be very choosy not just about their capabilities but about how well you work together, how easily you can make yourself understood, etc. If that fails, hire the smallest team that you can. Small is good because: Managing more people is more work. The relationship between number of people and management overhead is roughly O(n) but unevenly distributed; some people [...] --- First published: January 29th, 2026 Source: https://www.lesswrong.com/posts/cojSyfxfqfm4kpCbk/how-to-hire-a-team --- Narrated by TYPE III AUDIO.

The Possessed Machines is one of the most important AI microsites. It was published anonymously by an ex- lab employee, and does not seem to have spread very far, likely at least partly due to this anonymity (e.g. there is no LessWrong discussion at the time I'm posting this). This post is my attempt to fix that. I do not agree with everything in the piece, but I think cultural critiques of the "AGI uniparty" are vastly undersupplied and incredibly important in modeling & fixing the current trajectory. The piece is a long but worthwhile analysis of some of the cultural and psychological failures of the AGI industry. The frame is Dostoevsky's Demons (alternatively translated The Possessed), a novel about ruin in a small provincial town. The author argues it's best read as a detailed description of earnest people causing a catastrophe by following tracks laid down by the surrounding culture that have gotten corrupted: What I know is that Dostoevsky, looking at his own time, saw something true about how intelligent societies destroy themselves. He saw that the destruction comes from the best as well as the worst, from the idealists as well as the cynics, from the [...] --- First published: January 25th, 2026 Source: https://www.lesswrong.com/posts/ppBHrfY4bA6J7pkpS/the-possessed-machines-summary --- Narrated by TYPE III AUDIO.

Papal election of 1492 For over a decade, Ada Palmer, a history professor at University of Chicago (and a science-fiction writer!), struggled to teach Machiavelli. “I kept changing my approach, trying new things: which texts, what combinations, expanding how many class sessions he got…” The problem, she explains, is that “Machiavelli doesn’t unpack his contemporary examples, he assumes that you lived through it and know, so sometimes he just says things like: Some princes don’t have to work to maintain their power, like the Duke of Ferrara, period end of chapter. He doesn’t explain, so modern readers can’t get it.” Palmer's solution was to make her students live through the run-up to the Italian Wars themselves. Her current method involves a three-week simulation of the 1492 papal election, a massive undertaking with sixty students playing historical figures, each receiving over twenty pages of unique character material, supported by twenty chroniclers and seventy volunteers. After this almost month-long pedagogical marathon, a week of analysis, and reading Machiavelli's letters, students finally encounter The Prince. By then they know the context intimately. When Machiavelli mentions the Duke of Ferrara maintaining power effortlessly, Palmer's students react viscerally. They remember Alfonso and Ippolito d’Este as [...] --- First published: January 25th, 2026 Source: https://www.lesswrong.com/posts/doADJmyy6Yhp47SJ2/ada-palmer-inventing-the-renaissance --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a partial follow-up to AISLE discovered three new OpenSSL vulnerabilities from October 2025. TL;DR: OpenSSL is among the most scrutinized and audited cryptographic libraries on the planet, underpinning encryption for most of the internet. They just announced 12 new zero-day vulnerabilities (meaning previously unknown to maintainers at time of disclosure). We at AISLE discovered all 12 using our AI system. This is a historically unusual count and the first real-world demonstration of AI-based cybersecurity at this scale. Meanwhile, curl just cancelled its bug bounty program due to a flood of AI-generated spam, even as we reported 5 genuine CVEs to them. AI is simultaneously collapsing the median ("slop") and raising the ceiling (real zero-days in critical infrastructure). Background We at AISLE have been building an automated AI system for deep cybersecurity discovery and remediation, sometimes operating in bug bounties under the pseudonym Giant Anteater. Our goal was to turn what used to be an elite, artisanal hacker craft into a repeatable industrial process. We do this to secure the software infrastructure of human civilization before strong AI systems become ubiquitous. Prosaically, we want to make sure we don't get hacked into oblivion the moment they come online. [...] --- Outline: (01:05) Background (02:56) Fall 2025: Our first OpenSSL results (05:59) January 2026: 12 out of 12 new vulnerabilities (07:28) HIGH severity (1): (08:01) MODERATE severity (1): (08:24) LOW severity (10): (13:10) Broader impact: curl (17:06) The era of AI cybersecurity is here for good (18:40) Future outlook --- First published: January 27th, 2026 Source: https://www.lesswrong.com/posts/7aJwgbMEiKq5egQbd/ai-found-12-of-12-openssl-zero-days-while-curl-cancelled-its --- Narrated by TYPE III AUDIO.

Dario Amodei, CEO of Anthropic, has written a new essay on his thoughts on AI risk of various shapes. It seems worth reading, even if just for understanding what Anthropic is likely to do in the future. Confronting and Overcoming the Risks of Powerful AI There is a scene in the movie version of Carl Sagan's book Contact where the main character, an astronomer who has detected the first radio signal from an alien civilization, is being considered for the role of humanity's representative to meet the aliens. The international panel interviewing her asks, “If you could ask [the aliens] just one question, what would it be?” Her reply is: “I’d ask them, ‘How did you do it? How did you evolve, how did you survive this technological adolescence without destroying yourself?” When I think about where humanity is now with AI—about what we’re on the cusp of—my mind keeps going back to that scene, because the question is so apt for our current situation, and I wish we had the aliens’ answer to guide us. I believe we are entering a rite of passage, both turbulent and inevitable, which will test who we are as a species. Humanity [...] --- Outline: (00:24) Confronting and Overcoming the Risks of Powerful AI (15:19) 1. I'm sorry, Dave (15:23) Autonomy risks (28:53) Defenses (41:17) 2. A surprising and terrible empowerment (41:22) Misuse for destruction (54:50) Defenses (01:00:25) 3. The odious apparatus (01:00:30) Misuse for seizing power (01:13:08) Defenses (01:19:48) 4. Player piano (01:19:51) Economic disruption (01:21:18) Labor market disruption (01:33:43) Defenses (01:37:43) Economic concentration of power (01:40:49) Defenses (01:43:13) 5. Black seas of infinity (01:43:17) Indirect effects (01:47:29) Humanity's test (01:53:58) Footnotes The original text contained 92 footnotes which were omitted from this narration. --- First published: January 26th, 2026 Source: https://www.lesswrong.com/posts/kzPQohJakutbtFPcf/dario-amodei-the-adolescence-of-technology --- Narrated by TYPE III AUDIO.

Audio note: this article contains 78 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. This post covers work done by several researchers at, visitors to and collaborators of ARC, including Zihao Chen, George Robinson, David Matolcsi, Jacob Stavrianos, Jiawei Li and Michael Sklar. Thanks to Aryan Bhatt, Gabriel Wu, Jiawei Li, Lee Sharkey, Victor Lecomte and Zihao Chen for comments. In the wake of recent debate about pragmatic versus ambitious visions for mechanistic interpretability, ARC is sharing some models we've been studying that, in spite of their tiny size, serve as challenging test cases for any ambitious interpretability vision. The models are RNNs and transformers trained to perform algorithmic tasks, and range in size from 8 to 1,408 parameters. The largest model that we believe we more-or-less fully understand has 32 parameters; the next largest model that we have put substantial effort into, but have failed to fully understand, has 432 parameters. The models are available at the AlgZoo GitHub repo. We think that the "ambitious" side of the mechanistic interpretability community has historically underinvested in "fully understanding slightly complex [...] --- Outline: (03:09) Mechanistic estimates as explanations (06:16) Case study: 2nd argmax RNNs (08:30) Hidden size 2, sequence length 2 (14:47) Hidden size 4, sequence length 3 (16:13) Hidden size 16, sequence length 10 (19:52) Conclusion The original text contained 20 footnotes which were omitted from this narration. --- First published: January 26th, 2026 Source: https://www.lesswrong.com/posts/x8BbjZqooS4LFXS8Z/algzoo-uninterpreted-models-with-fewer-than-1-500-parameters --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

As soon as modern data analysis became a thing, the US government has had to deal with people trying to use open source data to uncover its secrets. During the early Cold War days and America's hydrogen bomb testing, there was an enormous amount of speculation about how the bombs actually worked. All nuclear technology involves refinement and purification of large amounts of raw substances into chemically pure substances. Armen Alchian was an economist working at RAND and reasoned that any US company working in such raw materials and supplying the government would have made a killing leading up to the tests. After checking financial data that RAND maintained on such companies, Alchian deduced that the secret sauce in the early fusion bombs was lithium and the Lithium Corporation of America was supplying the USG. The company's stock had skyrocketed leading up to the Castle Bravo test either by way of enormous unexpected revenue gains from government contracts, or more amusingly, maybe by government insiders buying up the stock trying to make a mushroom-cloud-sized fortune with the knowledge that lithium was the key ingredient. When word of this work got out, this story naturally ends with the FBI coming [...] --- Outline: (01:27) Pizza is the new lithium (03:09) The Data (04:11) The Backtest (04:36) Fordow bombing (04:55) Maduro capture (05:15) The Houthi stuff (10:25) Coda --- First published: January 22nd, 2026 Source: https://www.lesswrong.com/posts/Li3Aw7sDLXTCcQHZM/does-pentagon-pizza-theory-work --- Narrated by TYPE III AUDIO. --- Images from the article:

After five months of me (Buck) being slow at finishing up the editing on this, we’re finally putting out our inaugural Redwood Research podcast. I think it came out pretty well—we discussed a bunch of interesting and underdiscussed topics and I’m glad to have a public record of a bunch of stuff about our history. Tell your friends! Whether we do another one depends on how useful people find this one. You can watch on Youtube here, or as a Substack podcast. Notes on editing the podcast with Claude Code (Buck wrote this section) After the recording, we faced a problem. We had four hours of footage from our three cameras. We wanted it to snazzily cut between shots depending on who was talking. But I don’t truly in my heart believe that it's that important for the video editing to be that good, and I don’t really like the idea of paying a video editor. But I also don’t want to edit the four hours of video myself. And it seemed to me that video editing software was generally not optimized for the kind of editing I wanted to do here (especially automatically cutting between different shots according [...] --- Outline: (00:43) Notes on editing the podcast with Claude Code (03:11) Podcast transcript --- First published: January 4th, 2026 Source: https://www.lesswrong.com/posts/p4iJpumHt6Ay9KnXT/the-inaugural-redwood-research-podcast --- Narrated by TYPE III AUDIO.

This post was originally published on November 11th, 2025. I've been spending some time reworking and cleaning up the Inkhaven posts I'm most proud of, and completed the process for this one today. Today, Canada officially lost its measles elimination status. Measles was previously declared eliminated in Canada in 1998, but countries lose that status after 12 months of continuous transmission. Here are some articles about the the fact that we have lost our measles elimination status: CBC, BBC, New York Times, Toronto Life. You can see some chatter on Reddit about it if you're interested here. None of the above texts seemed to me to be focused on the actual thing that caused Canada to lose its measles elimination status, which is the rampant spread of measles among old-order religious communities, particularly the Mennonites. (Mennonites are basically, like, Amish-lite. Amish people can marry into Mennonite communities if they want a more laid-back lifestyle, but the reverse is not allowed. Similarly, old-order Mennonites can marry into less traditionally-minded Mennonite communities, but the reverse is not allowed.) The Reddit comments that made this point are generally not highly upvoted[1], and this was certainly not a central point in any of [...] --- Outline: (03:20) The Mennonite Outbreak (06:58) Mennonite Geography (11:47) Mennonites Are Susceptible To Facts and Logic, When Presented In Low German The original text contained 6 footnotes which were omitted from this narration. --- First published: January 25th, 2026 Source: https://www.lesswrong.com/posts/H8RdAbAmsqbpBWoDd/canada-lost-its-measles-elimination-status-because-we-don-t --- Narrated by TYPE III AUDIO. --- Images from the article:

Audio note: this article contains 73 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description. Epistemic status: This post is a synthesis of ideas that are, in my experience, widespread among researchers at frontier labs and in mechanistic interpretability, but rarely written down comprehensively in one place - different communities tend to know different pieces of evidence. The core hypothesis - that deep learning is performing something like tractable program synthesis - is not original to me (even to me, the ideas are ~3 years old), and I suspect it has been arrived at independently many times. (See the appendix on related work). This is also far from finished research - more a snapshot of a hypothesis that seems increasingly hard to avoid, and a case for why formalization is worth pursuing. I discuss the key barriers and how tools like singular learning theory might address them towards the end of the post. Thanks to Dan Murfet, Jesse Hoogland, Max Hennick, and Rumi Salazar for feedback on this post. Sam Altman: Why does unsupervised learning work? Dan Selsam: Compression. So, the ideal intelligence [...] --- Outline: (02:31) Background (09:06) Looking inside (09:09) Grokking (16:04) Vision circuits (22:37) The hypothesis (26:04) Why this isnt enough (27:22) Indirect evidence (32:44) The paradox of approximation (38:34) The paradox of generalization (45:44) The paradox of convergence (51:46) The path forward (53:20) The representation problem (58:38) The search problem (01:07:20) Appendix (01:07:23) Related work The original text contained 14 footnotes which were omitted from this narration. --- First published: January 20th, 2026 Source: https://www.lesswrong.com/posts/Dw8mskAvBX37MxvXo/deep-learning-as-program-synthesis-1 --- Narrated by TYPE III AUDIO. --- Images from the article:

Fiora Sunshine's post, Why I Transitioned: A Case Study (the OP) articulates a valuable theory for why some MtFs transition. If you are MtF and feel the post describes you, I believe you. However, many statements from the post are wrong or overly broad. My claims: There is evidence of a biological basis for trans identity. Twin studies are a good way to see this.   Fiora claims that trans people's apparent lack of introspective clarity may be evidence of deception. But trans people are incentivized not to attempt to share accurate answers to "why do you really want to transition?". This is the Trans Double Bind.   I am a counterexample to Fiora's theory. I was an adolescent social outcast weeb but did not transition. I spent 14 years actualizing as a man, then transitioned at 31 only after becoming crippled by dysphoria. My example shows that Fiora's phenotype can co-occur with or mask medically significant dysphoria. A. Biologically Transgender In the OP, Fiora presents the "body-map theory" under the umbrella of "arcane neuro-psychological phenomena", and then dismisses medical theories because the body-map theory doesn't fit her friend group. The body-map theory is a straw man for [...] --- Outline: (00:29) My claims: (01:17) A. Biologically Transgender (02:38) Twin Studies à la LLM (06:06) B. The Trans Double Bind (07:10) Motivations to transition (11:53) Introspective Clarity (13:23) C. In the Case of Quinoa Marisa (19:44) Conclusion The original text contained 11 footnotes which were omitted from this narration. --- First published: January 19th, 2026 Source: https://www.lesswrong.com/posts/rt2yai8JkTPYgzoEj/why-i-transitioned-a-response --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.