Advertise on podcast: AI: post transformers

This episode explores Context Language Models, a proposal that lets a model write directly to its own context as an editable file instead of relying on human-designed harnesses. It first surveys the existing approaches: fixed-policy compaction, action-based tools such as Context Folding, and recursive language models. It then shows why they fall short on tasks like in-place Sudoku edits, using a diagnostic benchmark that varies context pressure. The discussion explains why edits to the middle of a context break prefix caching and force costly recomputation, and why the paper measures cost in prefix-reuse FLOPs. It also lays out the paper's five claims: zero-shot gains on deep research and coding, swarms of agents with file-based contexts, evolved natural-language skills, online reinforcement learning, and a serving change called Suffix Cache Reuse. The hosts stress the difference between percentage points and relative percent, and they note that the strongest evidence is zero-shot with a large model while the training evidence is thinner. Sources: 1. Context Language Models — Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, Pang Wei Koh, 2026 http://arxiv.org/abs/2609.37725 2. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (introduces GRPO) — Zhihong Shao, Peiyi Wang, Qihao Zhu, et al. (DeepSeek-AI), 2024 https://scholar.google.com/scholar?q=DeepSeekMath%3A+Pushing+the+Limits+of+Mathematical+Reasoning+in+Open+Language+Models+%28introduces+GRPO%29 3. MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents — Zijian Zhou et al., 2025 https://scholar.google.com/scholar?q=MEM1%3A+Learning+to+Synergize+Memory+and+Reasoning+for+Efficient+Long-Horizon+Agents 4. Scaling Long-Horizon LLM Agent via Context-Folding — Weiwei Sun et al. (ByteDance Seed / CMU), 2025 https://scholar.google.com/scholar?q=Scaling+Long-Horizon+LLM+Agent+via+Context-Folding 5. BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent — Zijian Chen, Xueguang Ma, Shengyao Zhuang, et al. (Waterloo and collaborators), 2025 https://scholar.google.com/scholar?q=BrowseComp-Plus%3A+A+More+Fair+and+Transparent+Evaluation+Benchmark+of+Deep-Research+Agent 6. MemGPT: Towards LLMs as Operating Systems — Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez, 2023 https://scholar.google.com/scholar?q=MemGPT%3A+Towards+LLMs+as+Operating+Systems 7. How we built our multi-agent research system (Anthropic engineering report) / Effective context engineering for AI agents — Anthropic engineering team, 2025 https://scholar.google.com/scholar?q=How+we+built+our+multi-agent+research+system+%28Anthropic+engineering+report%29+%2F+Effective+context+engineering+for+AI+agents 8. Don't Build Multi-Agents (Cognition AI blog post) — Walden Yan (Cognition AI), 2025 https://scholar.google.com/scholar?q=Don%27t+Build+Multi-Agents+%28Cognition+AI+blog+post%29 9. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models (ACE) — Qizheng Zhang et al. (Stanford, SambaNova and collaborators), 2025 https://scholar.google.com/scholar?q=Agentic+Context+Engineering%3A+Evolving+Contexts+for+Self-Improving+Language+Models+%28ACE%29 10. ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization — Wu et al., 2025 https://scholar.google.com/scholar?q=ReSum%3A+Unlocking+Long-Horizon+Search+Intelligence+via+Context+Summarization 11. Training Language Models to Reason Efficiently — Arora and Zanette, 2025 (NeurIPS) https://scholar.google.com/scholar?q=Training+Language+Models+to+Reason+Efficiently 12. Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning (DDCA) — Peng et al., 2026 https://scholar.google.com/scholar?q=Think+Dense%2C+Not+Long%3A+Dynamic+Decoupled+Conditional+Advantage+for+Efficient+Reasoning+%28DDCA%29 13. CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents — Li, Hou, Jing, Tang, Dong, 2026 https://scholar.google.com/scholar?q=CompactionRL%3A+Reinforcement+Learning+with+Context+Compaction+for+Long-Horizon+Agents 14. Memento: Teaching LLMs to Manage Their Own Context — Kontonis et al., 2026 https://scholar.google.com/scholar?q=Memento%3A+Teaching+LLMs+to+Manage+Their+Own+Context 15. CacheBlend: Fast LLM Serving for RAG with Cached Knowledge Fusion; EPIC: Efficient Position-Independent Caching — Yao et al.; Hu et al., 2025 https://scholar.google.com/scholar?q=CacheBlend%3A+Fast+LLM+Serving+for+RAG+with+Cached+Knowledge+Fusion%3B+EPIC%3A+Efficient+Position-Independent+Caching 16. Recursive Language Models — Zhang, Kraska, Khattab, 2025 https://scholar.google.com/scholar?q=Recursive+Language+Models 17. Sculptor: Empowering LLMs with Cognitive Agency via Active Context Management — Li et al., 2026 https://scholar.google.com/scholar?q=Sculptor%3A+Empowering+LLMs+with+Cognitive+Agency+via+Active+Context+Management 18. Self-Generated Prompt Injections in Compaction Summaries — OpenAI, 2026 https://scholar.google.com/scholar?q=Self-Generated+Prompt+Injections+in+Compaction+Summaries 19. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning — Agrawal et al., 2026 https://scholar.google.com/scholar?q=GEPA%3A+Reflective+Prompt+Evolution+Can+Outperform+Reinforcement+Learning 20. Meta-Harness: End-to-End Optimization of Model Harnesses — Lee, Nair, Zhang, Lee, Khattab, Finn, 2026 https://scholar.google.com/scholar?q=Meta-Harness%3A+End-to-End+Optimization+of+Model+Harnesses Interactive Visualization: Context Language Models: Letting Models Edit Their Own Context

This episode examines "500xCompressor," a soft prompt compression method that squeezes hundreds of tokens of context into as little as a single learned token. A frozen Llama-3-8B decoder then answers questions from that token's key-value (KV) vectors alone. It covers why long contexts are costly (KV cache memory and attention cost grow with length). It also contrasts hard compression, which deletes text (Selective Context, LLMLingua), with soft approaches (gist tokens, AutoCompressor, ICAE, xRAG). The discussion walks through how a LoRA-adapted encoder and autoencoder-style regeneration pretraining produce those KV values, and what the headline 6x–480x ratios and 62–73% retained capability actually measure. The hosts also scrutinize the evaluation: arXiv abstracts from early 2024 are used to avoid pretraining leakage, and F1 versus Exact Match are explained. Listeners get a critical look at whether KV beats embeddings and whether the benchmark can be trusted. Sources: 1. 500xCompressor: Generalized Prompt Compression for Large Language Models — Zongqian Li, Yixuan Su, Nigel Collier, 2024 http://arxiv.org/abs/2408.03094v1 2. Compressing Context to Enhance Inference Efficiency of Large Language Models (Selective Context) — Li, Dong, Guerin, Lin, 2023 https://scholar.google.com/scholar?q=Compressing+Context+to+Enhance+Inference+Efficiency+of+Large+Language+Models+%28Selective+Context%29 3. Learning to Compress Prompts with Gist Tokens — Mu, Li, Goodman, 2023 https://scholar.google.com/scholar?q=Learning+to+Compress+Prompts+with+Gist+Tokens 4. Adapting Language Models to Compress Contexts (AutoCompressor) — Chevalier, Wettig, Ajith, Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts+%28AutoCompressor%29 5. xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token — Cheng et al., 2024 https://scholar.google.com/scholar?q=xRAG%3A+Extreme+Context+Compression+for+Retrieval-augmented+Generation+with+One+Token 6. In-context Autoencoder for Context Compression in a Large Language Model (ICAE) — Ge, Jing, Wang, Wang, Chen, Wei, 2024 https://scholar.google.com/scholar?q=In-context+Autoencoder+for+Context+Compression+in+a+Large+Language+Model+%28ICAE%29 7. Prefix-Tuning: Optimizing Continuous Prompts for Generation — Li, Liang, 2021 https://scholar.google.com/scholar?q=Prefix-Tuning%3A+Optimizing+Continuous+Prompts+for+Generation 8. Cartridges: Lightweight and general-purpose long context representations via self-study — Eyuboglu et al., 2025 https://scholar.google.com/scholar?q=Cartridges%3A+Lightweight+and+general-purpose+long+context+representations+via+self-study 9. Function Vectors in Large Language Models — Todd et al., 2024 https://scholar.google.com/scholar?q=Function+Vectors+in+Large+Language+Models 10. Efficient Streaming Language Models with Attention Sinks (StreamingLLM) — Xiao et al., 2024 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%28StreamingLLM%29 11. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhang et al., 2023 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models Interactive Visualization: 500xCompressor: Compressing Prompts into a Single Learned Token

This episode explores a Lean 4 formalization of Hopfield networks and Boltzmann machines, covering about sixteen thousand new lines on top of mathlib, which the authors say is the first for either model in any proof assistant. It walks through Hopfield energy descent and asynchronous convergence, Hebbian weights and the cross-talk term for non-orthogonal patterns, and Gibbs sampling as a random-scan Markov chain with detailed balance. It also covers the zero-temperature limit that recovers the Hopfield rule. The main finding is that formalizing convergence required every neuron to be updated infinitely often, a fairness condition the textbook theorem left implicit. The episode also points out that the Boltzmann machine formalized is fully visible, with no hidden units, so the ergodicity result covers less than the word suggests. It places the work against modern context, including the link between continuous Hopfield updates and softmax attention and dense associative memory. Listeners interested in what machine-checked proofs expose in classic results will find it useful. Sources: 1. Formalized Hopfield Networks and Boltzmann Machines — Matteo Cipollina, Michail Karatarakis, Freek Wiedijk, 2025 http://arxiv.org/abs/2512.07766 2. A Learning Algorithm for Boltzmann Machines — David H. Ackley, Geoffrey E. Hinton, Terrence J. Sejnowski, 1985 https://scholar.google.com/scholar?q=A+Learning+Algorithm+for+Boltzmann+Machines 3. Training Products of Experts by Minimizing Contrastive Divergence — Geoffrey E. Hinton, 2002 https://scholar.google.com/scholar?q=Training+Products+of+Experts+by+Minimizing+Contrastive+Divergence 4. A Fast Learning Algorithm for Deep Belief Nets — Geoffrey E. Hinton, Simon Osindero, Yee-Whye Teh, 2006 https://scholar.google.com/scholar?q=A+Fast+Learning+Algorithm+for+Deep+Belief+Nets 5. Deep Boltzmann Machines — Ruslan Salakhutdinov, Geoffrey E. Hinton, 2009 https://scholar.google.com/scholar?q=Deep+Boltzmann+Machines 6. The PageRank Citation Ranking: Bringing Order to the Web — Lawrence Page, Sergey Brin, Rajeev Motwani, Terry Winograd, 1998 https://scholar.google.com/scholar?q=The+PageRank+Citation+Ranking%3A+Bringing+Order+to+the+Web 7. Markov Chains and Mixing Times — David A. Levin, Yuval Peres, Elizabeth L. Wilmer, 2009 (2nd ed. 2017, with Propp) https://scholar.google.com/scholar?q=Markov+Chains+and+Mixing+Times 8. Non-negative Matrices and Markov Chains — Eugene Seneta, 1981 (2nd ed.; originally 1973) https://scholar.google.com/scholar?q=Non-negative+Matrices+and+Markov+Chains 9. Hopfield Networks is All You Need — Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, et al. (Sepp Hochreiter's group), 2020 https://scholar.google.com/scholar?q=Hopfield+Networks+is+All+You+Need 10. Dense Associative Memory for Pattern Recognition — Dmitry Krotov, John Hopfield, 2016 https://scholar.google.com/scholar?q=Dense+Associative+Memory+for+Pattern+Recognition 11. Neural networks and physical systems with emergent collective computational abilities — John J. Hopfield, 1982 https://scholar.google.com/scholar?q=Neural+networks+and+physical+systems+with+emergent+collective+computational+abilities 12. On Contrastive Divergence Learning — Miguel Á. Carreira-Perpiñán, Geoffrey E. Hinton, 2005 https://scholar.google.com/scholar?q=On+Contrastive+Divergence+Learning 13. Stochastic Ising machines and p-bits / probabilistic computing — Kerem Y. Camsari, Brian M. Sutton, Supriyo Datta (and related work), 2017-2019 https://scholar.google.com/scholar?q=Stochastic+Ising+machines+and+p-bits+%2F+probabilistic+computing 14. Formalizing Perron–Frobenius / Perron-Frobenius Theorem for Spectral Radius Analysis — René Thiemann, Jose Divasón, et al., 2016-2019 https://scholar.google.com/scholar?q=Formalizing+Perron%E2%80%93Frobenius+%2F+Perron-Frobenius+Theorem+for+Spectral+Radius+Analysis 15. Certigrad: Practical Verification of Stochastic Computation Graphs — Daniel Selsam, Percy Liang, David Dill, 2017 https://scholar.google.com/scholar?q=Certigrad%3A+Practical+Verification+of+Stochastic+Computation+Graphs 16. Rank collapse causes over-smoothing and over-squashing / Attention is not all you need: pure attention loses rank doubly exponentially with depth — Yihe Dong, Jean-Baptiste Cordonnier, Andreas Loukas, 2021 https://scholar.google.com/scholar?q=Rank+collapse+causes+over-smoothing+and+over-squashing+%2F+Attention+is+not+all+you+need%3A+pure+attention+loses+rank+doubly+exponentially+with+depth 17. Equation of State Calculations by Fast Computing Machines / Optimization by Simulated Annealing — Metropolis et al.; Kirkpatrick, Gelatt, Vecchi, 1953 / 1983 https://scholar.google.com/scholar?q=Equation+of+State+Calculations+by+Fast+Computing+Machines+%2F+Optimization+by+Simulated+Annealing Interactive Visualization: Formalizing Hopfield Networks and Boltzmann Machines in Lean 4

This episode examines a 2024 paper arguing that mixture-of-experts models outperform dense Transformers at every compute budget, with a gap that widens at scale. That claim contradicts earlier Meta and DeepMind studies, which saw the MoE advantage shrink, and the authors blame two hidden assumptions: a fixed training duration and a fixed expert size. The discussion builds the background from dense feed-forward layers, routers, and the difference between active and total parameters, through the Kaplan and Chinchilla scaling laws. It then introduces "granularity", the ratio of dense feed-forward width to expert width, as a third scaling axis. Granularity is a design choice that was inherited by convention and never shown to be optimal. The episode also stays skeptical of the evidence. The law is fit on a ragged grid of small models (3M to 85M active parameters), yet its headline claims extrapolate to a trillion active parameters, so listeners get a clear view of both the idea and its limits. Sources: 1. Scaling Laws for Fine-Grained Mixture of Experts — Jakub Krajewski, Jan Ludziejewski, Kamil Adamczewski, Maciej Pióro, Michał Krutul, Szymon Antoniak, Kamil Ciebiera, Krystian Król, Tomasz Odrzygóźdź, Piotr Sankowski, Marek Cygan, Sebastian Jaszczur, 2024 http://arxiv.org/abs/2402.07871 2. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models — Damai Dai, Chengqi Deng, Chenggang Zhao, et al. (DeepSeek-AI), 2024 https://scholar.google.com/scholar?q=DeepSeekMoE%3A+Towards+Ultimate+Expert+Specialization+in+Mixture-of-Experts+Language+Models 3. Scaling Laws for Fine-Grained Mixture of Experts — Jakub Krajewski, Jan Ludziejewski, Kamil Adamczewski, Maciej Pióro, Sebastian Jaszczur, et al., 2024 https://scholar.google.com/scholar?q=Scaling+Laws+for+Fine-Grained+Mixture+of+Experts 4. Unified Scaling Laws for Routed Language Models — Aidan Clark, Diego de las Casas, Aurelia Guy, et al. (DeepMind), 2022 https://scholar.google.com/scholar?q=Unified+Scaling+Laws+for+Routed+Language+Models 5. Mixture of A Million Experts — Xu Owen He (Google DeepMind), 2024 https://scholar.google.com/scholar?q=Mixture+of+A+Million+Experts 6. Scaling Laws for Neural Language Models — Jared Kaplan, Sam McCandlish, Tom Henighan, Tom Brown, et al. (OpenAI), 2020 https://scholar.google.com/scholar?q=Scaling+Laws+for+Neural+Language+Models 7. Training Compute-Optimal Large Language Models — Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al. (DeepMind), 2022 https://scholar.google.com/scholar?q=Training+Compute-Optimal+Large+Language+Models 8. Scaling Laws for Sparsely-Connected Foundation Models / Scaling Laws for MoE variants (e.g., Artetxe et al., 'Efficient Large Scale Language Modeling with Mixtures of Experts') — Mikel Artetxe, Shruti Bhosale, Naman Goyal, et al. (Meta AI), 2022 https://scholar.google.com/scholar?q=Scaling+Laws+for+Sparsely-Connected+Foundation+Models+%2F+Scaling+Laws+for+MoE+variants+%28e.g.%2C+Artetxe+et+al.%2C+%27Efficient+Large+Scale+Language+Modeling+with+Mixtures+of+Experts%27%29 9. Scaling Laws for Fine-Grained Mixture of Experts (ICML 2024 version) and follow-ups on joint scaling of E, G and D — Krajewski, Ludziejewski, Adamczewski, et al., 2024 https://scholar.google.com/scholar?q=Scaling+Laws+for+Fine-Grained+Mixture+of+Experts+%28ICML+2024+version%29+and+follow-ups+on+joint+scaling+of+E%2C+G+and+D 10. Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient — Ludziejewski, Pióro, Krajewski, et al., 2025 https://scholar.google.com/scholar?q=Joint+MoE+Scaling+Laws%3A+Mixture+of+Experts+Can+Be+Memory+Efficient 11. Efficient Large Scale Language Modeling with Mixtures of Experts — Mikel Artetxe, Shruti Bhosale, et al., 2022 https://scholar.google.com/scholar?q=Efficient+Large+Scale+Language+Modeling+with+Mixtures+of+Experts 12. Training Compute-Optimal Large Language Models (Chinchilla) — Jordan Hoffmann, Sebastian Borgeaud, et al., 2022 https://scholar.google.com/scholar?q=Training+Compute-Optimal+Large+Language+Models+%28Chinchilla%29 13. Mixture-of-Experts with Expert Choice Routing — Yanqi Zhou, Tao Lei, et al., 2022 https://scholar.google.com/scholar?q=Mixture-of-Experts+with+Expert+Choice+Routing 14. Mixture of A Million Experts (PEER) — Xu Owen He, 2024 https://scholar.google.com/scholar?q=Mixture+of+A+Million+Experts+%28PEER%29 15. Switch Transformers; GShard; Mixtral of Experts — Fedus, Zoph, Shazeer; Lepikhin et al.; Jiang et al., 2021-2024 https://scholar.google.com/scholar?q=Switch+Transformers%3B+GShard%3B+Mixtral+of+Experts 16. Transformer Feed-Forward Layers Are Key-Value Memories — Mor Geva, Roei Schuster, Jonathan Berant, Omer Levy, 2021 https://scholar.google.com/scholar?q=Transformer+Feed-Forward+Layers+Are+Key-Value+Memories 17. Scaling Laws for Sparsely-Connected Foundation Models — Elias Frantar, Carlos Riquelme, et al., 2023 https://scholar.google.com/scholar?q=Scaling+Laws+for+Sparsely-Connected+Foundation+Models 18. Scaling Laws for Upcycling Mixture-of-Experts Language Models; Scaling Laws for Sparsely-Activated models at inference — Liew et al.; related inference-cost work, 2025 https://scholar.google.com/scholar?q=Scaling+Laws+for+Upcycling+Mixture-of-Experts+Language+Models%3B+Scaling+Laws+for+Sparsely-Activated+models+at+inference Interactive Visualization: Scaling Laws for Fine-Grained Mixture of Experts

This episode examines "How Linear Attention Remembers," a preprint that asks how a single fact is written into, retained in, and read back out of the fixed-size recurrent state that replaces the KV cache. It contrasts Gated Linear Attention with Gated DeltaNet (whose delta rule overwrites old bindings instead of piling on top of them) and with hybrids that keep a few full-attention layers. The authors use associative-recall tests with causal interventions on pretrained models. They swap a head's state between near-identical "Paris" and "London" runs and measure the shift in the logit margin. Screening picks out one "hub" head per model, and the episode lays out the paper's four claims: content-specific writes, an identifiable read path, shared-state coupling that degrades recall as load grows, and attention layers taking over direct recall in hybrids. Listeners get the background on why a sum-based memory can't be pointed at the way a KV cache can, and a look at what that means for long-context serving on phones, edge devices, and robots. Sources: 1. How Linear Attention Remembers — Kichang Lee, JaeYeon Park, Songkuk Kim, JeongGil Ko, 2026 http://arxiv.org/abs/2609.33093 2. Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention — Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, Francois Fleuret, 2020 https://scholar.google.com/scholar?q=Transformers+are+RNNs%3A+Fast+Autoregressive+Transformers+with+Linear+Attention 3. Linear Transformers Are Secretly Fast Weight Programmers — Imanol Schlag, Kazuki Irie, Juergen Schmidhuber, 2021 https://scholar.google.com/scholar?q=Linear+Transformers+Are+Secretly+Fast+Weight+Programmers 4. Gated Delta Networks: Improving Mamba2 with Delta Rule — Songlin Yang, Jan Kautz, Ali Hatamizadeh, 2025 https://scholar.google.com/scholar?q=Gated+Delta+Networks%3A+Improving+Mamba2+with+Delta+Rule 5. Simple Linear Attention Language Models Balance the Recall-Throughput Tradeoff (Based) — Simran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina, Silas Alberti, Dylan Zinsley, James Zou, Atri Rudra, Christopher Re, 2024 https://scholar.google.com/scholar?q=Simple+Linear+Attention+Language+Models+Balance+the+Recall-Throughput+Tradeoff+%28Based%29 6. Repeat After Me: Transformers are Better than State Space Models at Copying — Samy Jelassi, David Brandfonbrener, Sham M. Kakade, Eran Malach, 2024 https://scholar.google.com/scholar?q=Repeat+After+Me%3A+Transformers+are+Better+than+State+Space+Models+at+Copying 7. Locating and Editing Factual Associations in GPT — Kevin Meng, David Bau, Alex Andonian, Yonatan Belinkov, 2022 https://scholar.google.com/scholar?q=Locating+and+Editing+Factual+Associations+in+GPT 8. Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small — Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, Jacob Steinhardt, 2022 https://scholar.google.com/scholar?q=Interpretability+in+the+Wild%3A+a+Circuit+for+Indirect+Object+Identification+in+GPT-2+small 9. In-context Learning and Induction Heads — Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, and others (Anthropic), 2022 https://scholar.google.com/scholar?q=In-context+Learning+and+Induction+Heads 10. The Hidden Attention of Mamba Models — Ameen Ali, Itamar Zimerman, Lior Wolf, 2024 https://scholar.google.com/scholar?q=The+Hidden+Attention+of+Mamba+Models 11. Zoology: Measuring and Improving Recall in Efficient Language Models — Simran Arora, Sabri Eyuboglu, Aman Timalsina, et al., 2023 https://scholar.google.com/scholar?q=Zoology%3A+Measuring+and+Improving+Recall+in+Efficient+Language+Models 12. Hopfield Networks is All You Need — Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, et al., 2020 https://scholar.google.com/scholar?q=Hopfield+Networks+is+All+You+Need 13. Learning to (Learn at Test Time): RNNs with Expressive Hidden States — Yu Sun, Xinhao Li, Karan Dalal, et al., 2024 https://scholar.google.com/scholar?q=Learning+to+%28Learn+at+Test+Time%29%3A+RNNs+with+Expressive+Hidden+States 14. Mamba: Linear-Time Sequence Modeling with Selective State Spaces / Transformers are SSMs (Mamba-2) — Albert Gu, Tri Dao, 2023/2024 https://scholar.google.com/scholar?q=Mamba%3A+Linear-Time+Sequence+Modeling+with+Selective+State+Spaces+%2F+Transformers+are+SSMs+%28Mamba-2%29 15. Locating and Editing Factual Associations in Mamba — Arnab Sen Sharma, David Atkinson, David Bau, 2024 https://scholar.google.com/scholar?q=Locating+and+Editing+Factual+Associations+in+Mamba 16. Mamba-Based Language Models: An Empirical Study (Nemotron-H / An Empirical Study of Mamba-based Language Models) — Roger Waleffe, Wonmin Byeon, et al., 2024 https://scholar.google.com/scholar?q=Mamba-Based+Language+Models%3A+An+Empirical+Study+%28Nemotron-H+%2F+An+Empirical+Study+of+Mamba-based+Language+Models%29 17. Investigating the Indirect Object Identification circuit in GPT-2 small / Interpretability in the Wild — Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, Jacob Steinhardt, 2022 https://scholar.google.com/scholar?q=Investigating+the+Indirect+Object+Identification+circuit+in+GPT-2+small+%2F+Interpretability+in+the+Wild 18. Zamba / Jamba / Samba hybrid papers (e.g., Jamba: A Hybrid Transformer-Mamba Language Model) — Opher Lieber et al. (Jamba); Liliang Ren et al. (Samba), 2024 https://scholar.google.com/scholar?q=Zamba+%2F+Jamba+%2F+Samba+hybrid+papers+%28e.g.%2C+Jamba%3A+A+Hybrid+Transformer-Mamba+Language+Model%29 19. Understanding the Differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks (Test-time regression view) — Jerome Sieber, Carmen Amo Alonso, et al. / Ke Alexander Wang, Jiaxin Shi, Emily B. Fox (Test-time regression), 2024/2025 https://scholar.google.com/scholar?q=Understanding+the+Differences+in+Foundation+Models%3A+Attention%2C+State+Space+Models%2C+and+Recurrent+Neural+Networks+%28Test-time+regression+view%29 20. Towards Best Practices of Activation Patching in Language Models: Metrics and Methods — Fred Zhang, Neel Nanda, 2024 https://scholar.google.com/scholar?q=Towards+Best+Practices+of+Activation+Patching+in+Language+Models%3A+Metrics+and+Methods Interactive Visualization: How Linear Attention Remembers: Writing and Recalling Facts

This episode examines "Mixture of A Million Experts," a 2024 paper proposing a learned router that selects among more than a million single-neuron experts at sublinear cost. It explains why conventional mixture-of-experts routing stalls around 128 experts: scoring every expert against every token scales linearly. It also covers why fixed hash routers fall short. The discussion traces the paper's building blocks: fine-grained MoE scaling laws that favor ever-smaller experts, Product Key Memory's Cartesian-product trick for cheap retrieval, and the view of feedforward layers as key-value memories. Along the way, the hosts flag caveats, including the paper's redefinition of granularity, the extrapolation of a scaling law far beyond where it was fitted, and lifelong learning being motivation only, with no experiment. They also raise whether FLOP-based isoFLOP wins will hold up on real hardware, where scattered lookups into a million-entry table are a poor fit for large matrix multiplies. Sources: 1. Mixture of A Million Experts — Xu Owen He, 2024 http://arxiv.org/abs/2407.04153 2. Scaling Laws for Fine-Grained Mixture of Experts — Jakub Krajewski, Jan Ludziejewski, Kamil Adamczewski, Maciej Pióro, Michał Krutul, Szymon Antoniak, Kamil Ciebiera, Krystian Król, Tomasz Odrzygóźdź, Piotr Sankowski, Marek Cygan, Sebastian Jaszczur, 2024 https://scholar.google.com/scholar?q=Scaling+Laws+for+Fine-Grained+Mixture+of+Experts 3. Unified Scaling Laws for Routed Language Models — Aidan Clark, Diego de las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake Hechtman, Trevor Cai, Sebastian Borgeaud, George van den Driessche, Eliza Rutherford, Tom Hennigan, Matthew Johnson, Katie Millican, Albin Cassirer, Chris Jones, Elena Buchatskaya, David Budden, Laurent Sifre, Simon Osindero, Oriol Vinyals, Jack Rae, Erich Elsen, Koray Kavukcuoglu, Karen Simonyan, 2022 https://scholar.google.com/scholar?q=Unified+Scaling+Laws+for+Routed+Language+Models 4. Mixture of A Million Experts (PEER) — Xu Owen He, 2024 https://scholar.google.com/scholar?q=Mixture+of+A+Million+Experts+%28PEER%29 5. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models — Damai Dai, Chengqi Deng, Chenggang Zhao, R.X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, Wenfeng Liang, 2024 https://scholar.google.com/scholar?q=DeepSeekMoE%3A+Towards+Ultimate+Expert+Specialization+in+Mixture-of-Experts+Language+Models 6. Large Memory Layers with Product Keys — Guillaume Lample, Alexandre Sablayrolles, Marc'Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou, 2019 https://scholar.google.com/scholar?q=Large+Memory+Layers+with+Product+Keys 7. Transformer Feed-Forward Layers Are Key-Value Memories — Mor Geva, Roei Schuster, Jonathan Berant, Omer Levy, 2021 https://scholar.google.com/scholar?q=Transformer+Feed-Forward+Layers+Are+Key-Value+Memories 8. Mixture-of-Experts with Expert Choice Routing — Yanqi Zhou et al., 2022 https://scholar.google.com/scholar?q=Mixture-of-Experts+with+Expert+Choice+Routing 9. Memory augmented language models through mixture of word experts — Cicero Nogueira dos Santos et al., 2023 https://scholar.google.com/scholar?q=Memory+augmented+language+models+through+mixture+of+word+experts 10. Fast Feedforward Networks — Peter Belcak, Roger Wattenhofer, 2023 https://scholar.google.com/scholar?q=Fast+Feedforward+Networks 11. Approximating Two-Layer Feedforward Networks for Efficient Transformers — Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber, 2023 https://scholar.google.com/scholar?q=Approximating+Two-Layer+Feedforward+Networks+for+Efficient+Transformers 12. Lifelong Language Pretraining with Distribution-Specialized Experts (Lifelong-MoE) — Wuyang Chen et al., 2023 https://scholar.google.com/scholar?q=Lifelong+Language+Pretraining+with+Distribution-Specialized+Experts+%28Lifelong-MoE%29 13. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer — Noam Shazeer et al., 2017 https://scholar.google.com/scholar?q=Outrageously+Large+Neural+Networks%3A+The+Sparsely-Gated+Mixture-of-Experts+Layer Interactive Visualization: Mixture of a Million Experts: Scaling Routing with Product Keys

This episode examines "HyperTuning," an ICML 2023 paper that uses a hypernetwork (HyperT5) to adapt a frozen language model to a new task in a single forward pass, with no back-propagation through the downstream model. It explains how the hypermodel reads up to sixteen labeled examples and emits prefix or LoRA parameters that the downstream model reuses for every query without seeing the examples again. The discussion covers why per-task adaptation is costly, how parameter-efficient methods still require a full backward pass, and how the work relates to earlier hypernetwork, meta-learning, and instruction-to-parameter approaches. It also sets up a skeptical reading of the paper's three claims: that HyperT5 roughly matches full fine-tuning on held-out tasks, that it beats shared-parameter PEFT baselines, and that its output works as a PEFT initialization. The skeptical questions are where the training cost actually moves to, and whether the headline margins hold beyond single-run noise. Sources: 1. HyperTuning: Adapting Language Models Without Back-propagation Using Hypernetworks https://proceedings.mlr.press/v202/phang23a/phang23a.pdf 2. Multitask Prompted Training Enables Zero-Shot Task Generalization (T0) — Victor Sanh, Albert Webson, Colin Raffel, et al., 2022 (ICLR; arXiv 2021) https://scholar.google.com/scholar?q=Multitask+Prompted+Training+Enables+Zero-Shot+Task+Generalization+%28T0%29 3. Finetuned Language Models Are Zero-Shot Learners (FLAN) — Jason Wei, Maarten Bosma, Vincent Zhao, et al., 2022 (ICLR; arXiv 2021) https://scholar.google.com/scholar?q=Finetuned+Language+Models+Are+Zero-Shot+Learners+%28FLAN%29 4. Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks — Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, et al., 2022 https://scholar.google.com/scholar?q=Super-NaturalInstructions%3A+Generalization+via+Declarative+Instructions+on+1600%2B+NLP+Tasks 5. MetaICL: Learning to Learn In Context — Sewon Min, Mike Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi, 2022 https://scholar.google.com/scholar?q=MetaICL%3A+Learning+to+Learn+In+Context 6. HyperNetworks — David Ha, Andrew Dai, Quoc V. Le, 2016 https://scholar.google.com/scholar?q=HyperNetworks 7. HINT: Hypernetwork Instruction Tuning for Efficient Zero-Shot Generalisation — Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew Peters, 2022 https://scholar.google.com/scholar?q=HINT%3A+Hypernetwork+Instruction+Tuning+for+Efficient+Zero-Shot+Generalisation 8. Boosting Natural Language Generation from Instructions with Meta-Learning — Budhaditya Deb, Guoqing Zheng, Ahmed Hassan Awadallah, 2022 https://scholar.google.com/scholar?q=Boosting+Natural+Language+Generation+from+Instructions+with+Meta-Learning 9. Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks (HyperFormer) — Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, James Henderson, 2021 https://scholar.google.com/scholar?q=Parameter-efficient+Multi-task+Fine-tuning+for+Transformers+via+Shared+Hypernetworks+%28HyperFormer%29 10. SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer — Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, Daniel Cer, 2021 https://scholar.google.com/scholar?q=SPoT%3A+Better+Frozen+Model+Adaptation+through+Soft+Prompt+Transfer 11. Reducing Retraining by Recycling Parameter-Efficient Prompts — Brian Lester, Joshua Yurtsever, Siamak Shakeri, Noah Constant, 2022 https://scholar.google.com/scholar?q=Reducing+Retraining+by+Recycling+Parameter-Efficient+Prompts 12. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML) — Chelsea Finn, Pieter Abbeel, Sergey Levine, 2017 https://scholar.google.com/scholar?q=Model-Agnostic+Meta-Learning+for+Fast+Adaptation+of+Deep+Networks+%28MAML%29 13. Hypernetworks and meta-learned weight generators for LLMs (e.g. Text-to-LoRA, Drag-and-Drop LLMs) — Sakana AI (Text-to-LoRA); Liang et al. (Drag-and-Drop LLMs), 2025 https://scholar.google.com/scholar?q=Hypernetworks+and+meta-learned+weight+generators+for+LLMs+%28e.g.+Text-to-LoRA%2C+Drag-and-Drop+LLMs%29 14. Transformers generalize differently from information stored in context vs in weights — Stephanie Chan, Ishita Dasgupta, Junkyung Kim, Dharshan Kumaran, Andrew Lampinen, Felix Hill, 2022 https://scholar.google.com/scholar?q=Transformers+generalize+differently+from+information+stored+in+context+vs+in+weights Interactive Visualization: HyperTuning: Adapting Language Models Without Back-propagation Using Hypernetworks

This episode examines "Learning to Compress Prompts with Gist Tokens," which trains a language model to squeeze an instruction into a few cached gist-token activations. The trick is a modified attention mask: tokens after the gist can't see the original prompt, so its information has to pass through the gist positions. The discussion covers why the KV cache and per-task methods like prefix tuning, prompt tuning, and context distillation don't fully solve the cost of reused prompts. It frames gisting as a hypernetwork that predicts the compressed prefix in one forward pass, trained with ordinary instruction tuning and no extra objective. It also questions the headline 26x compression figure. That number is a ratio of averages from prompts of about 26 tokens, which may not support the long system-prompt use case the paper opens with. Listeners get a clear explanation of the mechanism and a skeptical look at what the evidence actually shows. Sources: 1. Learning to Compress Prompts with Gist Tokens — Jesse Mu, Xiang Lisa Li, Noah Goodman, 2023 http://arxiv.org/abs/2304.08467 2. Learning to Compress Prompts with Gist Tokens — Jesse Mu, Xiang Lisa Li, Noah Goodman, 2023 (NeurIPS 2023; arXiv 2304.08467) https://scholar.google.com/scholar?q=Learning+to+Compress+Prompts+with+Gist+Tokens 3. The Power of Scale for Parameter-Efficient Prompt Tuning — Brian Lester, Rami Al-Rfou, Noah Constant, 2021 https://scholar.google.com/scholar?q=The+Power+of+Scale+for+Parameter-Efficient+Prompt+Tuning 4. Prefix-Tuning: Optimizing Continuous Prompts for Generation — Xiang Lisa Li, Percy Liang, 2021 https://scholar.google.com/scholar?q=Prefix-Tuning%3A+Optimizing+Continuous+Prompts+for+Generation 5. Adapting Language Models to Compress Contexts (AutoCompressors) — Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts+%28AutoCompressors%29 6. In-context Autoencoder for Context Compression in a Large Language Model (ICAE) — Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, Furu Wei, 2023 https://scholar.google.com/scholar?q=In-context+Autoencoder+for+Context+Compression+in+a+Large+Language+Model+%28ICAE%29 7. Language Modeling with Learned Summary or Memory Tokens: Compressed Context Memory for Online Language Model Interaction — Jang-Hyun Kim, Junyoung Yeom, Sangdoo Yun, Hyun Oh Song, 2023 https://scholar.google.com/scholar?q=Language+Modeling+with+Learned+Summary+or+Memory+Tokens%3A+Compressed+Context+Memory+for+Online+Language+Model+Interaction 8. Efficient Streaming Language Models with Attention Sinks (StreamingLLM) — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%28StreamingLLM%29 9. Longformer: The Long-Document Transformer — Iz Beltagy, Matthew E. Peters, Arman Cohan, 2020 https://scholar.google.com/scholar?q=Longformer%3A+The+Long-Document+Transformer 10. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 https://scholar.google.com/scholar?q=LLMLingua%3A+Compressing+Prompts+for+Accelerated+Inference+of+Large+Language+Models 11. Learning to Compress Contexts / Compressive Transformers for Long-Range Sequence Modelling — Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Chloe Hillier, Timothy P. Lillicrap, 2020 https://scholar.google.com/scholar?q=Learning+to+Compress+Contexts+%2F+Compressive+Transformers+for+Long-Range+Sequence+Modelling 12. Learning by Distilling Context — Charlie Snell, Dan Klein, Ruiqi Zhong, 2022 https://scholar.google.com/scholar?q=Learning+by+Distilling+Context 13. Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models — David Wingate, Mohammad Shoeybi, Taylor Sorensen, 2022 https://scholar.google.com/scholar?q=Prompt+Compression+and+Contrastive+Conditioning+for+Controllability+and+Toxicity+Reduction+in+Language+Models 14. Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) — Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, et al., 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention+%28vLLM%29 15. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints — Joshua Ainslie, James Lee-Thorp, Michiel de Jong, et al., 2023 https://scholar.google.com/scholar?q=GQA%3A+Training+Generalized+Multi-Query+Transformer+Models+from+Multi-Head+Checkpoints 16. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, et al., 2023 https://scholar.google.com/scholar?q=Judging+LLM-as-a-Judge+with+MT-Bench+and+Chatbot+Arena Interactive Visualization: Gist Tokens: Compressing Prompts Up to 26x

This episode examines "Investigating the Effectiveness of HyperTuning via Gisting," which asks whether a few-shot examples can be compressed into just sixteen tokens of state so a frozen Llama-2 adapts to a task without backpropagation, instead of re-reading the examples. It sets up the cost trade-offs of in-context learning (examples re-read on every request), parameter-efficient fine-tuning (training needed per task), and hypernetworks (amortized one-time training that generates adapters in a single forward pass). The central insight is that Gist tokens' key and value states are effectively a soft prefix, so a Gisting model can act as a prefix generator using only an attention-mask change, with no dedicated parameter-generation heads or expensive hyperpretraining stage like the earlier HyperT5. The episode also describes the HyperLlama design, a single 4-bit quantized Llama-2-7B that plays hypernetwork or frozen model depending on whether its LoRA adapter is on. Listeners get a clear explanation of how the approach is built, why it is cheap, and why its author reports only mixed performance against simply reading the examples. Sources: 1. Investigating the Effectiveness of HyperTuning via Gisting — Jason Phang, 2024 http://arxiv.org/abs/2402.16817 2. HyperNetworks — David Ha, Andrew M. Dai, Quoc V. Le, 2017 (ICLR 2017) https://scholar.google.com/scholar?q=HyperNetworks 3. HyperTuning: Toward Adapting Large Language Models without Back-propagation — Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen, 2023 (ICML 2023) https://scholar.google.com/scholar?q=HyperTuning%3A+Toward+Adapting+Large+Language+Models+without+Back-propagation 4. Learning to Compress Prompts with Gist Tokens — Jesse Mu, Xiang Lisa Li, Noah Goodman, 2023 (NeurIPS 2023) https://scholar.google.com/scholar?q=Learning+to+Compress+Prompts+with+Gist+Tokens 5. HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation — Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew E. Peters, 2023 (ACL 2023) https://scholar.google.com/scholar?q=HINT%3A+Hypernetwork+Instruction+Tuning+for+Efficient+Zero-+%26+Few-Shot+Generalisation 6. HyperTuning: Toward Adapting Large Language Models without Backpropagation — Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen, 2023 https://scholar.google.com/scholar?q=HyperTuning%3A+Toward+Adapting+Large+Language+Models+without+Backpropagation 7. Adapting Language Models to Compress Contexts (AutoCompressors) — Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts+%28AutoCompressors%29 8. In-context Autoencoder for Context Compression in a Large Language Model (ICAE) — Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, Furu Wei, 2023 https://scholar.google.com/scholar?q=In-context+Autoencoder+for+Context+Compression+in+a+Large+Language+Model+%28ICAE%29 9. Prefix-Tuning: Optimizing Continuous Prompts for Generation — Xiang Lisa Li, Percy Liang, 2021 https://scholar.google.com/scholar?q=Prefix-Tuning%3A+Optimizing+Continuous+Prompts+for+Generation 10. Transformers Learn In-Context by Gradient Descent — Johannes von Oswald et al., 2023 https://scholar.google.com/scholar?q=Transformers+Learn+In-Context+by+Gradient+Descent 11. Symbol tuning improves in-context learning in language models — Jerry Wei et al., 2023 https://scholar.google.com/scholar?q=Symbol+tuning+improves+in-context+learning+in+language+models 12. Cartridges: Lightweight and General-Purpose Long Context Representations via Self-Study — Sabri Eyuboglu et al., 2025 https://scholar.google.com/scholar?q=Cartridges%3A+Lightweight+and+General-Purpose+Long+Context+Representations+via+Self-Study 13. Text-to-LoRA: Instant Transformer Adaption — Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Lange, 2025 https://scholar.google.com/scholar?q=Text-to-LoRA%3A+Instant+Transformer+Adaption 14. Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights — Zhiyuan Liang et al., 2025 https://scholar.google.com/scholar?q=Drag-and-Drop+LLMs%3A+Zero-Shot+Prompt-to-Weights 15. Hypernetworks (HyperNetworks) — David Ha, Andrew Dai, Quoc Le, 2017 https://scholar.google.com/scholar?q=Hypernetworks+%28HyperNetworks%29 16. 500xCompressor: Generalized Prompt Compression for Large Language Models — Zongqian Li, Yixuan Su, Nigel Collier, 2024 https://scholar.google.com/scholar?q=500xCompressor%3A+Generalized+Prompt+Compression+for+Large+Language+Models Interactive Visualization: HyperTuning via Gisting: Cheap Adapter Generation, Mixed Results

This episode explores a 2021 formal proof, in the Lean theorem prover, that decision stumps (one-threshold classifiers) are PAC learnable. It starts from a simple learner that places its threshold at the largest positively labeled training point, so it can only err by calling true positives negative. The discussion explains why formalizing this on the real line is hard. Probability over uncountable spaces needs sigma-algebras and measurability, which textbooks skip, and the Giry monad lets the formal statement stay close to the pencil-and-paper one. It also compares this work with earlier formal ML efforts, including Certigrad's proof of unbiased stochastic backpropagation gradients in Lean, Coq-verified Hoeffding bounds for finite hypothesis classes, and VC-dimension theory, which the paper deliberately leaves out. The episode also sets up the claim that a step in the standard textbook proofs doesn't hold as stated. That claim should interest anyone curious about what machine-checked proofs reveal about familiar learning-theory results. Sources: 1. A Formal Proof of PAC Learnability for Decision Stumps — Joseph Tassarotti, Koundinya Vajjha, Anindya Banerjee, Jean-Baptiste Tristan, 2019 http://arxiv.org/abs/1911.00385 2. A Formal Proof of PAC Learnability for Decision Stumps — Joseph Tassarotti, Koundinya Vajjha, Anindya Banerjee, Jean-Baptiste Tristan, 2021 (CPP '21; arXiv 1911.00385) https://scholar.google.com/scholar?q=A+Formal+Proof+of+PAC+Learnability+for+Decision+Stumps 3. Developing Bug-Free Machine Learning Systems With Formal Mathematics — Daniel Selsam, Percy Liang, David L. Dill, 2017 (ICML) https://scholar.google.com/scholar?q=Developing+Bug-Free+Machine+Learning+Systems+With+Formal+Mathematics 4. Formal Verification of Neural Network Robustness and Related Surveys (e.g., Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks) — Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer, 2017 (CAV) https://scholar.google.com/scholar?q=Formal+Verification+of+Neural+Network+Robustness+and+Related+Surveys+%28e.g.%2C+Reluplex%3A+An+Efficient+SMT+Solver+for+Verifying+Deep+Neural+Networks%29 5. Formal Verification of a Stochastic Gradient Descent Convergence Proof / Formal Verification of Stochastic Gradient Descent (Lean and Coq lines of work) — Various, including Koundinya Vajjha, Avraham Shinnar, Barry Trager, Vincent Pestun, Nathan Fulton (Coq), 2021-2022 https://scholar.google.com/scholar?q=Formal+Verification+of+a+Stochastic+Gradient+Descent+Convergence+Proof+%2F+Formal+Verification+of+Stochastic+Gradient+Descent+%28Lean+and+Coq+lines+of+work%29 6. A Theory of the Learnable — Leslie G. Valiant, 1984 https://scholar.google.com/scholar?q=A+Theory+of+the+Learnable 7. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting — Yoav Freund, Robert E. Schapire, 1997 https://scholar.google.com/scholar?q=A+Decision-Theoretic+Generalization+of+On-Line+Learning+and+an+Application+to+Boosting 8. Greedy Function Approximation: A Gradient Boosting Machine — Jerome H. Friedman, 2001 https://scholar.google.com/scholar?q=Greedy+Function+Approximation%3A+A+Gradient+Boosting+Machine 9. Understanding Machine Learning: From Theory to Algorithms — Shai Shalev-Shwartz, Shai Ben-David, 2014 https://scholar.google.com/scholar?q=Understanding+Machine+Learning%3A+From+Theory+to+Algorithms 10. Interactive Theorem Proving and Program Development: Coq'Art / Formalization of measure theory and Lebesgue integration in Isabelle/HOL — Johannes Hölzl and Armin Heller, 2011 https://scholar.google.com/scholar?q=Interactive+Theorem+Proving+and+Program+Development%3A+Coq%27Art+%2F+Formalization+of+measure+theory+and+Lebesgue+integration+in+Isabelle%2FHOL 11. Formalization of Lebesgue Integration and Probability in Coq / Verified probabilistic reasoning (e.g., Coq-based measure theory by Affeldt, Cohen and collaborators) — Reynald Affeldt, Cyril Cohen, and collaborators, 2019-2023 https://scholar.google.com/scholar?q=Formalization+of+Lebesgue+Integration+and+Probability+in+Coq+%2F+Verified+probabilistic+reasoning+%28e.g.%2C+Coq-based+measure+theory+by+Affeldt%2C+Cohen+and+collaborators%29 12. Formalizing Probability Theory in Lean's mathlib — Mathlib community, including Kexing Ying, Rémy Degenne and others, 2020-2024 https://scholar.google.com/scholar?q=Formalizing+Probability+Theory+in+Lean%27s+mathlib 13. Machine Learning and Measure Theory textbook treatment: Foundations of Machine Learning / Foundations of Modern Probability — Mohri, Rostamizadeh, Talwalkar; Olav Kallenberg, 2018 / 2002 https://scholar.google.com/scholar?q=Machine+Learning+and+Measure+Theory+textbook+treatment%3A+Foundations+of+Machine+Learning+%2F+Foundations+of+Modern+Probability 14. The Lean Theorem Prover (System Description) — Leonardo de Moura, Soonho Kong, Jeremy Avigad, Floris van Doorn, Jakob von Raumer, 2015 (CADE) https://scholar.google.com/scholar?q=The+Lean+Theorem+Prover+%28System+Description%29 15. The Lean 4 Theorem Prover and Programming Language — Leonardo de Moura and Sebastian Ullrich, 2021 (CADE) https://scholar.google.com/scholar?q=The+Lean+4+Theorem+Prover+and+Programming+Language 16. Formal Verification of a Realistic Compiler (CompCert) — Xavier Leroy, 2009 https://scholar.google.com/scholar?q=Formal+Verification+of+a+Realistic+Compiler+%28CompCert%29 17. seL4: Formal Verification of an OS Kernel — Gerwin Klein et al., 2009 https://scholar.google.com/scholar?q=seL4%3A+Formal+Verification+of+an+OS+Kernel 18. Certified Programming with Dependent Types / Developing Bug-Free Machine Learning Systems With Formal Mathematics (Certigrad) — Daniel Selsam, Percy Liang, David L. Dill, 2017 https://scholar.google.com/scholar?q=Certified+Programming+with+Dependent+Types+%2F+Developing+Bug-Free+Machine+Learning+Systems+With+Formal+Mathematics+%28Certigrad%29 19. Certifying the True Error: Machine Learning in Coq with Verified Generalization Guarantees — Alexander Bagnall, Gordon Stewart, 2019 https://scholar.google.com/scholar?q=Certifying+the+True+Error%3A+Machine+Learning+in+Coq+with+Verified+Generalization+Guarantees 20. Learnability and the Vapnik-Chervonenkis Dimension — Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, Manfred K. Warmuth, 1989 https://scholar.google.com/scholar?q=Learnability+and+the+Vapnik-Chervonenkis+Dimension 21. Verified Analysis of Random Trees — Manuel Eberl, Max W. Haslbeck, Tobias Nipkow, 2018 https://scholar.google.com/scholar?q=Verified+Analysis+of+Random+Trees 22. Formal-ML: formal machine learning library for Lean — Martin Zinkevich, 2020 https://scholar.google.com/scholar?q=Formal-ML%3A+formal+machine+learning+library+for+Lean 23. A Formal Proof of the Expressiveness of Deep Learning — Alexander Bentkamp, Jasmin Christian Blanchette, Dietrich Klakow, 2019 https://scholar.google.com/scholar?q=A+Formal+Proof+of+the+Expressiveness+of+Deep+Learning 24. A Formally Verified Proof of the Central Limit Theorem — Jeremy Avigad, Johannes Hölzl, Luke Serafin, 2017 https://scholar.google.com/scholar?q=A+Formally+Verified+Proof+of+the+Central+Limit+Theorem 25. Verified Tail Bounds for Randomized Programs — Joseph Tassarotti, Robert Harper, 2018 https://scholar.google.com/scholar?q=Verified+Tail+Bounds+for+Randomized+Programs 26. Formal verification of generalization / PAC-Bayes and Rademacher complexity bounds in Lean 4 / mathlib (2023-2025 follow-up work) — Various (Lean/mathlib community; Zinkevich-style learning-theory libraries; Lean 4 'LeanMachineLearning' efforts), 2023-2025 https://scholar.google.com/scholar?q=Formal+verification+of+generalization+%2F+PAC-Bayes+and+Rademacher+complexity+bounds+in+Lean+4+%2F+mathlib+%282023-2025+follow-up+work%29 Interactive Visualization: Formal Lean Proof of PAC Learnability for Decision Stumps

This episode examines "Verifying Peephole Rewriting In SSA Compiler IRs," which uses the Lean proof assistant to give MLIR-style compilers a push-button workflow for proving small rewrites correct, without the cost of a fully verified compiler. It explains peephole rewrites and why SSA def-use chains let a matcher catch patterns that straight-line matching misses. It then compares the approach with Alive and Alive2, which rely on SMT solvers, and with CompCert-style end-to-end verification, which takes years of expert effort and is tied to one IR. The discussion covers MLIR dialects and regions, and why a ring-aware prover helps for domains like fully homomorphic encryption, where an algebraically trivial rewrite becomes a heroic proof at the LLVM level. It also covers refinement semantics with poison and undefined behavior. The episode adds a caveat from Csmith's results: proving local rewrites correct does not cover bugs in parsing, lowering, or pass ordering. Listeners get a clear picture of where formal compiler verification can reach into new domain-specific IRs and where it stops. Sources: 1. Verifying Peephole Rewriting In SSA Compiler IRs — Siddharth Bhat, Alex Keizer, Chris Hughes, Andrés Goens, Tobias Grosser, 2024 http://arxiv.org/abs/2407.03685 2. Peephole Optimization — William M. McKeeman, 1965 https://scholar.google.com/scholar?q=Peephole+Optimization 3. Provably Correct Peephole Optimizations with Alive — Nuno P. Lopes, David Menendez, Santosh Nagarakatte, John Regehr, 2015 https://scholar.google.com/scholar?q=Provably+Correct+Peephole+Optimizations+with+Alive 4. Alive2: Bounded Translation Validation for LLVM — Nuno P. Lopes, Juneyoung Lee, Chung-Kil Hur, Zhengyang Liu, John Regehr, 2021 https://scholar.google.com/scholar?q=Alive2%3A+Bounded+Translation+Validation+for+LLVM 5. Verified Peephole Optimizations for CompCert — Eric Mullen, Daryl Zuniga, Zachary Tatlock, Dan Grossman, 2016 https://scholar.google.com/scholar?q=Verified+Peephole+Optimizations+for+CompCert 6. MLIR: Scaling Compiler Infrastructure for Domain Specific Computation — Chris Lattner, Mehdi Amini, Uday Bondhugula, Albert Cohen, Andy Davis, Jacques Pienaar, River Riddle, Tatiana Shpeisman, Nicolas Vasilache, Oleksandr Zinenko, 2021 https://scholar.google.com/scholar?q=MLIR%3A+Scaling+Compiler+Infrastructure+for+Domain+Specific+Computation 7. Static Single Assignment Form and the Control Dependence Graph — Ron Cytron, Jeanne Ferrante, Barry K. Rosen, Mark N. Wegman, F. Kenneth Zadeck, 1991 https://scholar.google.com/scholar?q=Static+Single+Assignment+Form+and+the+Control+Dependence+Graph 8. Compiling Lean programs with MLIR / LEAN to MLIR (lz and lean-mlir) — Siddharth Bhat, Tobias Grosser and collaborators, 2022 https://scholar.google.com/scholar?q=Compiling+Lean+programs+with+MLIR+%2F+LEAN+to+MLIR+%28lz+and+lean-mlir%29 9. Verifying Peephole Rewriting In SSA Compiler IRs — Siddharth Bhat, Alex Keizer, Chris Hughes, Andrés Goens, Tobias Grosser, 2024 https://scholar.google.com/scholar?q=Verifying+Peephole+Rewriting+In+SSA+Compiler+IRs 10. Formal Verification of a Realistic Compiler — Xavier Leroy, 2009 https://scholar.google.com/scholar?q=Formal+Verification+of+a+Realistic+Compiler 11. Finding and Understanding Bugs in C Compilers — Xuejun Yang, Yang Chen, Eric Eide, John Regehr, 2011 https://scholar.google.com/scholar?q=Finding+and+Understanding+Bugs+in+C+Compilers 12. CakeML: A Verified Implementation of ML — Ramana Kumar, Magnus O. Myreen, Michael Norrish, Scott Owens, 2014 https://scholar.google.com/scholar?q=CakeML%3A+A+Verified+Implementation+of+ML 13. Formalizing the LLVM Intermediate Representation for Verified Program Transformations (Vellvm) — Jianzhou Zhao, Santosh Nagarakatte, Milo M. K. Martin, Steve Zdancewic, 2012 https://scholar.google.com/scholar?q=Formalizing+the+LLVM+Intermediate+Representation+for+Verified+Program+Transformations+%28Vellvm%29 14. Taming Undefined Behavior in LLVM — Juneyoung Lee, Yoonseung Kim, Youngju Song, Chung-Kil Hur, Sanjoy Das, David Majnemer, John Regehr, Nuno P. Lopes, 2017 https://scholar.google.com/scholar?q=Taming+Undefined+Behavior+in+LLVM 15. Bringing the WebAssembly Standard up to Speed with SpecTec / Mechanising and Verifying the WebAssembly Specification — Conrad Watt (and collaborators including Andreas Rossberg), 2018 https://scholar.google.com/scholar?q=Bringing+the+WebAssembly+Standard+up+to+Speed+with+SpecTec+%2F+Mechanising+and+Verifying+the+WebAssembly+Specification 16. Finding and Understanding Bugs in C Compilers (Csmith) — Xuejun Yang, Yang Chen, Eric Eide, John Regehr, 2011 https://scholar.google.com/scholar?q=Finding+and+Understanding+Bugs+in+C+Compilers+%28Csmith%29 17. Formal Verification of a Realistic Compiler (CompCert) — Xavier Leroy, 2009 https://scholar.google.com/scholar?q=Formal+Verification+of+a+Realistic+Compiler+%28CompCert%29 18. Vellvm: Formalizing the LLVM Intermediate Representation for Verified Program Transformations — Jianzhou Zhao, Santosh Nagarakatte, Milo M. K. Martin, Steve Zdancewic, 2012 https://scholar.google.com/scholar?q=Vellvm%3A+Formalizing+the+LLVM+Intermediate+Representation+for+Verified+Program+Transformations 19. Interaction Trees: Representing Recursive and Impure Programs in Coq — Li-yao Xia, Yannick Zakowski, Paul He, Chung-Kil Hur, Gregory Malecha, Benjamin C. Pierce, Steve Zdancewic, 2020 https://scholar.google.com/scholar?q=Interaction+Trees%3A+Representing+Recursive+and+Impure+Programs+in+Coq 20. Souper: A Synthesizing Superoptimizer — Raimondas Sasnauskas, Yang Chen, Peter Collingbourne, Jeroen Ketema, Gratian Lup, Jubi Taneja, John Regehr, 2017 https://scholar.google.com/scholar?q=Souper%3A+A+Synthesizing+Superoptimizer 21. TASO: Optimizing Deep Learning Computation with Automatic Generation of Graph Substitutions — Zhihao Jia, Oded Padon, James Thomas, Todd Warszawski, Matei Zaharia, Alex Aiken, 2019 https://scholar.google.com/scholar?q=TASO%3A+Optimizing+Deep+Learning+Computation+with+Automatic+Generation+of+Graph+Substitutions 22. PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated Corrections — Haojie Wang, Jidong Zhai, Mingyu Gao, Zixuan Ma, et al., 2021 https://scholar.google.com/scholar?q=PET%3A+Optimizing+Tensor+Programs+with+Partially+Equivalent+Transformations+and+Automated+Corrections 23. egg: Fast and Extensible Equality Saturation — Max Willsey, Chandrakana Nandi, Yisu Remy Wang, Oliver Flatt, Zachary Tatlock, Pavel Panchekha, 2021 https://scholar.google.com/scholar?q=egg%3A+Fast+and+Extensible+Equality+Saturation 24. Verified Compilation of Neural Network / Tensor Programs (Alive-tv for MLIR) — Seongwon Bang, Seunghyeon Nam, Inwhan Chun, Ho Young Jhoo, Juneyoung Lee, 2022 https://scholar.google.com/scholar?q=Verified+Compilation+of+Neural+Network+%2F+Tensor+Programs+%28Alive-tv+for+MLIR%29 25. Learning to Superoptimize Programs / LLM-guided compiler optimization (e.g., Meta LLM Compiler) — Rudy Bunel et al.; Chris Cummins et al., 2016 / 2024 https://scholar.google.com/scholar?q=Learning+to+Superoptimize+Programs+%2F+LLM-guided+compiler+optimization+%28e.g.%2C+Meta+LLM+Compiler%29 Interactive Visualization: Verifying Peephole Rewrites in SSA Compiler IRs with Lean

This episode explores the Saxe, McClelland, and Ganguli paper on exact learning dynamics in deep linear neural networks. It explains why these networks are worth studying even though they collapse to a single linear map: their gradient-descent dynamics are still coupled and non-convex, and they show the long plateaus followed by sudden error drops seen in real deep nets. The discussion covers the Baldi and Hornik result that every critical point is a saddle except the best low-rank solution. It then covers the "decoupled invariant manifold," where each input-output mode evolves independently. The mode-strength equation, whose growth rate is proportional to its own value, explains why small initial weights produce long plateaus. The episode argues that depth need not slow learning if the initial conditions are chosen well, and ties this to unsupervised pretraining and to random initialization. Listeners get a clear, step-by-step derivation of why training time depends on the starting point. Sources: 1. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks — Andrew M. Saxe, James L. McClelland, Surya Ganguli, 2013 http://arxiv.org/abs/1312.6120 2. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks — Andrew M. Saxe, James L. McClelland, Surya Ganguli, 2013 (arXiv:1312.6120; ICLR 2014) https://scholar.google.com/scholar?q=Exact+solutions+to+the+nonlinear+dynamics+of+learning+in+deep+linear+neural+networks 3. Deep Learning without Poor Local Minima — Kenji Kawaguchi, 2016 https://scholar.google.com/scholar?q=Deep+Learning+without+Poor+Local+Minima 4. On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization — Sanjeev Arora, Nadav Cohen, Elad Hazan, 2018 https://scholar.google.com/scholar?q=On+the+Optimization+of+Deep+Networks%3A+Implicit+Acceleration+by+Overparameterization 5. Implicit Regularization in Deep Matrix Factorization — Sanjeev Arora, Nadav Cohen, Wei Hu, Yuping Luo, 2019 https://scholar.google.com/scholar?q=Implicit+Regularization+in+Deep+Matrix+Factorization 6. A Fast Learning Algorithm for Deep Belief Nets — Geoffrey E. Hinton, Simon Osindero, Yee-Whye Teh, 2006 https://scholar.google.com/scholar?q=A+Fast+Learning+Algorithm+for+Deep+Belief+Nets 7. Greedy Layer-Wise Training of Deep Networks — Yoshua Bengio, Pascal Lamblin, Dan Popovici, Hugo Larochelle, 2006 (NIPS 2007) https://scholar.google.com/scholar?q=Greedy+Layer-Wise+Training+of+Deep+Networks 8. Why Does Unsupervised Pre-training Help Deep Learning? — Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, Samy Bengio, 2010 (JMLR) https://scholar.google.com/scholar?q=Why+Does+Unsupervised+Pre-training+Help+Deep+Learning%3F 9. Understanding the difficulty of training deep feedforward neural networks — Xavier Glorot, Yoshua Bengio, 2010 https://scholar.google.com/scholar?q=Understanding+the+difficulty+of+training+deep+feedforward+neural+networks 10. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification — Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, 2015 https://scholar.google.com/scholar?q=Delving+Deep+into+Rectifiers%3A+Surpassing+Human-Level+Performance+on+ImageNet+Classification 11. Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks — Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, Jeffrey Pennington, 2018 https://scholar.google.com/scholar?q=Dynamical+Isometry+and+a+Mean+Field+Theory+of+CNNs%3A+How+to+Train+10%2C000-Layer+Vanilla+Convolutional+Neural+Networks 12. Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice — Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli, 2017 https://scholar.google.com/scholar?q=Resurrecting+the+sigmoid+in+deep+learning+through+dynamical+isometry%3A+theory+and+practice 13. Exponential expressivity in deep neural networks through transient chaos — Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, Surya Ganguli, 2016 https://scholar.google.com/scholar?q=Exponential+expressivity+in+deep+neural+networks+through+transient+chaos 14. Deep Information Propagation — Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein, 2017 https://scholar.google.com/scholar?q=Deep+Information+Propagation 15. Neural Tangents: Fast and Easy Infinite Neural Networks in Python — Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, Samuel S. Schoenholz, 2019 https://scholar.google.com/scholar?q=Neural+Tangents%3A+Fast+and+Easy+Infinite+Neural+Networks+in+Python 16. Learning hierarchical category structure in deep neural networks — A. M. Saxe, J. L. McClelland, S. Ganguli, 2013 https://scholar.google.com/scholar?q=Learning+hierarchical+category+structure+in+deep+neural+networks 17. Neural networks and principal component analysis: Learning from examples without local minima — P. Baldi, K. Hornik, 1989 https://scholar.google.com/scholar?q=Neural+networks+and+principal+component+analysis%3A+Learning+from+examples+without+local+minima 18. Effect of Batch Learning In Multilayer Neural Networks — K. Fukumizu, 1998 https://scholar.google.com/scholar?q=Effect+of+Batch+Learning+In+Multilayer+Neural+Networks 19. On the importance of initialization and momentum in deep learning — I. Sutskever, J. Martens, G. Dahl, G. Hinton, 2013 https://scholar.google.com/scholar?q=On+the+importance+of+initialization+and+momentum+in+deep+learning 20. Exact learning dynamics of deep linear networks with prior knowledge — L. Braun, C. Dominé, J. Fitzgerald, A. Saxe, 2022 https://scholar.google.com/scholar?q=Exact+learning+dynamics+of+deep+linear+networks+with+prior+knowledge 21. Deep learning without poor local minima (and related: Kawaguchi 2016) — K. Kawaguchi, 2016 https://scholar.google.com/scholar?q=Deep+learning+without+poor+local+minima+%28and+related%3A+Kawaguchi+2016%29 22. A mathematical theory of semantic development in deep neural networks — A. Saxe, J. McClelland, S. Ganguli, 2019 https://scholar.google.com/scholar?q=A+mathematical+theory+of+semantic+development+in+deep+neural+networks 23. Implicit regularization in deep matrix factorization / Arora, Cohen, Hu, Luo — S. Arora, N. Cohen, W. Hu, Y. Luo, 2019 https://scholar.google.com/scholar?q=Implicit+regularization+in+deep+matrix+factorization+%2F+Arora%2C+Cohen%2C+Hu%2C+Luo 24. On the Convergence of Gradient Descent for Wide/Deep Linear Networks (Arora, Cohen, Golowich, Hu) — S. Arora, N. Cohen, N. Golowich, W. Hu, 2019 https://scholar.google.com/scholar?q=On+the+Convergence+of+Gradient+Descent+for+Wide%2FDeep+Linear+Networks+%28Arora%2C+Cohen%2C+Golowich%2C+Hu%29 25. Progress measures for grokking via mechanistic interpretability / Saddle-to-saddle dynamics literature (e.g., Jacot et al., 'Saddle-to-saddle dynamics in deep linear networks') — A. Jacot, F. Ged, B. Şimşek, C. Hongler, F. Gabriel, 2021 https://scholar.google.com/scholar?q=Progress+measures+for+grokking+via+mechanistic+interpretability+%2F+Saddle-to-saddle+dynamics+literature+%28e.g.%2C+Jacot+et+al.%2C+%27Saddle-to-saddle+dynamics+in+deep+linear+networks%27%29 26. Deep Residual Learning for Image Recognition — K. He, X. Zhang, S. Ren, J. Sun, 2016 https://scholar.google.com/scholar?q=Deep+Residual+Learning+for+Image+Recognition 27. Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (muP) — G. Yang, E. Hu et al., 2022 https://scholar.google.com/scholar?q=Tensor+Programs+V%3A+Tuning+Large+Neural+Networks+via+Zero-Shot+Hyperparameter+Transfer+%28muP%29 Interactive Visualization: Exact Learning Dynamics in Deep Linear Neural Networks

This episode explores a 2024 paper that tries to turn mechanistic interpretability into formal guarantees. The idea is to reverse-engineer a small transformer into an algorithm, then use that algorithm to write a much shorter proof of a lower bound on the model's accuracy. The test case is a one-layer, attention-only model trained to output the maximum of four integers. The discussion contrasts this approach with local-robustness verifiers and PAC-Bayes bounds. It also sets out the two trivial extremes: brute-force enumeration of about sixteen million inputs, and a vacuous accuracy bound of zero. The interpretation finds that the model attends to larger tokens, copies what it attends to, and barely depends on position. Treating everything the interpretation leaves unexplained as worst-case yields a "cubic" proof that cuts the compute roughly four-million-fold, at the cost of certifying about 95% accuracy instead of the true 99.9%. Listeners get a clear look at how a better mechanistic understanding could make guarantees cheaper, along with the caveat that the evidence comes from a single tiny model. Sources: 1. Compact Proofs of Model Performance via Mechanistic Interpretability — Jason Gross, Rajashree Agrawal, Thomas Kwa, Euan Ong, Chun Hei Yip, Alex Gibson, Soufiane Noubir, Lawrence Chan, 2024 http://arxiv.org/abs/2406.11779 2. Compact Proofs of Model Performance via Mechanistic Interpretability — Jason Gross, Rajashree Agrawal, Thomas Kwa, Euan Ong, Chun Hei Yip, Alex Gibson, Soufiane Noubir, Lawrence Chan, 2024 (arXiv 2406.11779; later versions through Dec 2024) https://scholar.google.com/scholar?q=Compact+Proofs+of+Model+Performance+via+Mechanistic+Interpretability 3. Progress measures for grokking via mechanistic interpretability — Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, Jacob Steinhardt, 2023 (ICLR 2023) https://scholar.google.com/scholar?q=Progress+measures+for+grokking+via+mechanistic+interpretability 4. A Mathematical Framework for Transformer Circuits — Nelson Elhage, Neel Nanda, Catherine Olsson, Chris Olah and others (Anthropic), 2021 https://scholar.google.com/scholar?q=A+Mathematical+Framework+for+Transformer+Circuits 5. Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems — David 'davidad' Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, Joshua Tenenbaum, 2024 https://scholar.google.com/scholar?q=Towards+Guaranteed+Safe+AI%3A+A+Framework+for+Ensuring+Robust+and+Reliable+AI+Systems 6. Formal Algorithms/Verification baselines: Towards Fast Computation of Certified Robustness for ReLU Networks (Fast-Lin) and Efficient Formal Safety Analysis of Neural Networks (Neurify); also Reluplex — Reluplex: Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer (2017); Fast-Lin: Tsui-Wei Weng et al. (2018); Neurify: Shiqi Wang et al. (2018), 2017-2018 https://scholar.google.com/scholar?q=Formal+Algorithms%2FVerification+baselines%3A+Towards+Fast+Computation+of+Certified+Robustness+for+ReLU+Networks+%28Fast-Lin%29+and+Efficient+Formal+Safety+Analysis+of+Neural+Networks+%28Neurify%29%3B+also+Reluplex 7. Formalizing the Presumption of Independence — Paul Christiano, Eric Neyman, Mark Xu, 2022 https://scholar.google.com/scholar?q=Formalizing+the+Presumption+of+Independence 8. Causal Scrubbing: a method for rigorously testing interpretability hypotheses — Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, Nate Thomas, 2022 https://scholar.google.com/scholar?q=Causal+Scrubbing%3A+a+method+for+rigorously+testing+interpretability+hypotheses 9. Mathematical Models of Computation in Superposition — Kaarel Hänni, Jake Mendel, Dmitry Vaintrob, Lawrence Chan, 2024 https://scholar.google.com/scholar?q=Mathematical+Models+of+Computation+in+Superposition 10. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks — Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer, 2017 https://scholar.google.com/scholar?q=Reluplex%3A+An+Efficient+SMT+Solver+for+Verifying+Deep+Neural+Networks 11. Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data — Gintare Karolina Dziugaite, Daniel M. Roy, 2017 https://scholar.google.com/scholar?q=Computing+Nonvacuous+Generalization+Bounds+for+Deep+%28Stochastic%29+Neural+Networks+with+Many+More+Parameters+than+Training+Data 12. AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation — Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, Martin Vechev, 2018 https://scholar.google.com/scholar?q=AI2%3A+Safety+and+Robustness+Certification+of+Neural+Networks+with+Abstract+Interpretation 13. How Transformers Learn Causal Structure with Gradient Descent — Eshaan Nichani, Alex Damian, Jason D. Lee, 2024 https://scholar.google.com/scholar?q=How+Transformers+Learn+Causal+Structure+with+Gradient+Descent 14. Opening the AI Black Box: Program Synthesis via Mechanistic Interpretability — Eric J. Michaud, Isaac Liao, Vedang Lad, Ziming Liu, Anish Mudide, Chloe Loughridge, et al., 2024 https://scholar.google.com/scholar?q=Opening+the+AI+Black+Box%3A+Program+Synthesis+via+Mechanistic+Interpretability Interactive Visualization: Compact Proofs of Model Performance via Mechanistic Interpretability

This episode examines "Backprop as Functor," which argues that gradient descent on a whole network equals the composition of the gradient-descent learners of its parts. It defines a learner as four pieces: a parameter space, an implementation, an update, and a "request" function. The request tells an upstream module what input it should have received, and this supplies the missing intermediate training target. The discussion explains the categorical machinery in plain terms (categories, functors, monoidal structure, and the Para and Learn categories). It presents the main theorem: a faithful, strong symmetric monoidal functor sends parametrised functions to gradient-descent learners, with the request given by backprop. The episode compares this to difference target propagation, which only approximates backprop, and to lenses and open games, where the request plays the role of the backward "put". It notes that the paper has no experiments and asks whether the formalism does more than restate the fact that the chain rule is functorial. It then starts a worked quadratic-error example showing why the composite update matches plain gradient descent. Listeners get a clear view of what a compositional treatment of backprop does and does not buy. Sources: 1. Backprop as Functor: A compositional perspective on supervised learning — Brendan Fong, David I. Spivak, Rémy Tuyéras, 2017 http://arxiv.org/abs/1711.10455 2. Backprop as Functor: A compositional perspective on supervised learning — Brendan Fong, David Spivak, Rémy Tuyéras, 2017 (arXiv); LICS 2019 https://scholar.google.com/scholar?q=Backprop+as+Functor%3A+A+compositional+perspective+on+supervised+learning 3. Categorical Foundations of Gradient-Based Learning — G.S.H. Cruttwell, Bruno Gavranović, Neil Ghani, Paul Wilson, Fabio Zanasi, 2021 (arXiv); ESOP 2022 https://scholar.google.com/scholar?q=Categorical+Foundations+of+Gradient-Based+Learning 4. The Simple Essence of Automatic Differentiation — Conal Elliott, 2018 https://scholar.google.com/scholar?q=The+Simple+Essence+of+Automatic+Differentiation 5. Position: Categorical Deep Learning is an Algebraic Theory of All Architectures — Bruno Gavranović, Paul Lessard, Andrew Dudzik, Tamara von Glehn, João G. M. Araújo, Petar Veličković, 2024 https://scholar.google.com/scholar?q=Position%3A+Categorical+Deep+Learning+is+an+Algebraic+Theory+of+All+Architectures 6. Difference Target Propagation — Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, Yoshua Bengio, 2015 https://scholar.google.com/scholar?q=Difference+Target+Propagation 7. Compositional Game Theory — Neil Ghani, Jules Hedges, Viktor Winschel, Philipp Zahn, 2016 (arXiv); LICS 2018 https://scholar.google.com/scholar?q=Compositional+Game+Theory 8. Combinators for Bidirectional Tree Transformations: A Linguistic Approach to the View-Update Problem — J. Nathan Foster, Michael B. Greenwald, Jonathan T. Moore, Benjamin C. Pierce, Alan Schmitt, 2007 https://scholar.google.com/scholar?q=Combinators+for+Bidirectional+Tree+Transformations%3A+A+Linguistic+Approach+to+the+View-Update+Problem 9. Lenses and Learners — Brendan Fong, Michael Johnson, 2019 https://scholar.google.com/scholar?q=Lenses+and+Learners 10. From Open Learners to Open Games — Jules Hedges, 2019 https://scholar.google.com/scholar?q=From+Open+Learners+to+Open+Games 11. Categorical Deep Learning: An Algebraic Theory of Architectures — Bruno Gavranović, Paul Lessard, Andrew Dudzik, Tamara von Glehn, João G. M. Araújo, Petar Veličković, 2024 https://scholar.google.com/scholar?q=Categorical+Deep+Learning%3A+An+Algebraic+Theory+of+Architectures 12. Reverse Derivative Categories — R. Cockett, G. Cruttwell, J. Gallagher, J.-S. P. Lemay, B. MacAdam, G. Plotkin, D. Pronk, 2020 https://scholar.google.com/scholar?q=Reverse+Derivative+Categories 13. Backpropagation in the Simply Typed Lambda-calculus with Linear Negation — Alois Brunel, Damiano Mazza, Michele Pagani, 2020 https://scholar.google.com/scholar?q=Backpropagation+in+the+Simply+Typed+Lambda-calculus+with+Linear+Negation 14. Understanding Deep Image Representations by Inverting Them — A. Mahendran, A. Vedaldi, 2015 https://scholar.google.com/scholar?q=Understanding+Deep+Image+Representations+by+Inverting+Them 15. Automatic Differentiation in Machine Learning: a Survey — A. G. Baydin, B. A. Pearlmutter, A. A. Radul, J. M. Siskind, 2018 https://scholar.google.com/scholar?q=Automatic+Differentiation+in+Machine+Learning%3A+a+Survey Interactive Visualization: Backprop as Functor: Composing Learners Through Category Theory

This episode examines DeLM, a Stanford system that runs several coding agents in parallel with no main agent, and asks whether four agents deliver four agents' worth of progress or just a bigger bill. It sorts existing designs by the "bubble" each one creates. Independent sampling wastes effort on redundant work. Peer debate stalls at round barriers and grows quadratically in messages. Centralized orchestrators like Claude Code and Codex leave the lead agent idle and pass lossy summaries between agents. DeLM replaces the lead with a self-service task queue, where the first claim wins, and an append-only shared log of FACT and FAIL entries that agents publish as they learn things. The discussion ties this to the 1980 Hearsay-II blackboard system, which had a scheduler where DeLM has none, and notes the new risk that stale or wrong natural-language findings can mislead peers. The hosts also say how they will judge the claimed accuracy and speed gains: whether the benchmarks are representative, whether the gaps exceed run-to-run noise, and how much of the speedup comes from the design rather than from idle baselines. Sources: 1. Decentralized Multi-Agent Systems with Shared Context — Yuzhen Mao, Jerry Gu, Aadi Chauhan, Qizheng Zhang, Hangoo Kang, Azalia Mirhoseini, 2026 http://arxiv.org/abs/2606.10662 2. The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty — Lee D. Erman, Frederick Hayes-Roth, Victor R. Lesser, D. Raj Reddy, 1980 https://scholar.google.com/scholar?q=The+Hearsay-II+Speech-Understanding+System%3A+Integrating+Knowledge+to+Resolve+Uncertainty 3. Generative Communication in Linda — David Gelernter, 1985 https://scholar.google.com/scholar?q=Generative+Communication+in+Linda 4. Improving Factuality and Reasoning in Language Models through Multiagent Debate — Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, Igor Mordatch, 2023 https://scholar.google.com/scholar?q=Improving+Factuality+and+Reasoning+in+Language+Models+through+Multiagent+Debate 5. Exploring Advanced LLM Multi-Agent Systems Based on Blackboard Architecture — Bochen Han, Songmao Zhang, 2025 https://scholar.google.com/scholar?q=Exploring+Advanced+LLM+Multi-Agent+Systems+Based+on+Blackboard+Architecture 6. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? — Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan, 2024 (ICLR; arXiv 2023) https://scholar.google.com/scholar?q=SWE-bench%3A+Can+Language+Models+Resolve+Real-World+GitHub+Issues%3F 7. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press, 2024 https://scholar.google.com/scholar?q=SWE-agent%3A+Agent-Computer+Interfaces+Enable+Automated+Software+Engineering 8. Introducing SWE-bench Verified — OpenAI (Preparedness team) with human annotators, in collaboration with the SWE-bench authors, 2024 https://scholar.google.com/scholar?q=Introducing+SWE-bench+Verified 9. Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — Mike A. Merrill, Alexander G. Shaw, and the Terminal-Bench team (Stanford and Laude Institute); the DeLM paper cites Merrill et al., 2026, 2025-2026 https://scholar.google.com/scholar?q=Terminal-Bench%3A+Benchmarking+Agents+on+Hard%2C+Realistic+Tasks+in+Command+Line+Interfaces 10. Large Language Monkeys: Scaling Inference Compute with Repeated Sampling — Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, Azalia Mirhoseini, 2024 https://scholar.google.com/scholar?q=Large+Language+Monkeys%3A+Scaling+Inference+Compute+with+Repeated+Sampling 11. Scheduling Multithreaded Computations by Work Stealing — Robert D. Blumofe, Charles E. Leiserson, 1999 https://scholar.google.com/scholar?q=Scheduling+Multithreaded+Computations+by+Work+Stealing 12. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism — Yanping Huang et al., 2019 https://scholar.google.com/scholar?q=GPipe%3A+Efficient+Training+of+Giant+Neural+Networks+using+Pipeline+Parallelism 13. AOrchestra — Ruan et al., 2026 https://scholar.google.com/scholar?q=AOrchestra 14. SPRINT: Enabling Interleaved Planning and Parallelized Execution in Reasoning Models — Emil Biju, Shayan Talaei, Zhemin Huang, Mohammadreza Pourreza, Azalia Mirhoseini, Amin Saberi, 2026 https://scholar.google.com/scholar?q=SPRINT%3A+Enabling+Interleaved+Planning+and+Parallelized+Execution+in+Reasoning+Models 15. ToolOrchestra — Su et al., 2025 https://scholar.google.com/scholar?q=ToolOrchestra 16. Recursive Language Models — Zhang et al., 2025 https://scholar.google.com/scholar?q=Recursive+Language+Models