Already Clean
Send us Fan Mail 📖 Read Why AI's "tangled black box" may have been a blurry microscope, and how clean, branching neurons change the story For ten years, the people who study artificial intelligence have been telling us the same story. Inside every large neural network, they said, there is a knot. Concepts crammed into the same few neurons, jammed together to save space, so snarled that no human could ever read them. We believed it because it felt true. Of course the machine is a mess. We built something enormous and strange, fed it the entire internet, and asked it to figure things out. Why would it be tidy? Now one of the researchers who helped tell that story has come back to say: look again. Reference Neural Networks Are Already Clean, by Nick Cammarata | Goodfire Chapters 00:00 Welcome to Heliox and Thanks to PRX Stations 02:03 Today’s Mission: Neural Networks Are Already Clean 02:53 The Superposition Myth Explained 04:46 The Dead Ends of Mechanistic Interpretability 09:46 Sparse Autoencoders and the 2019 Car-in-Dog Puzzle 11:51 The Broken Microscope: Feature Visualization 14:45 The Missing Express Lane: Residual Streams 17:42 The V-Shape Revelation 19:42 The Negative Pole: Round vs. Pointed 21:43 Recursive Generalization: The Tree Metaphor 22:47 The IY Neuron: From Letters to “Do It Yourself” 25:55 The Planetary Geology Neuron 27:49 One Learning Principle: The Branching Rainforest 28:41 Enter the Grassmannian Manifold 33:20 Geodesics and Canonical Angles 35:41 Proving It with Canonical Correlation Analysis 38:27 The Noisy Tail Is Nuance 40:04 The Missing Curves Stress Test 43:16 Do Language Models Hold Up? 44:52 The Colorado Comma Neuron 47:05 Who Stole the Book? Scar and Santa 50:22 The Ecology of Circuits and Gradient Starvation 52:25 The Umbrella-Into-a-Bee Experiment 54:16 Riemannian Optimization and Retractions 57:45 Why Networks Are Already Clean 58:50 The Final Question: Is Human Thought Geometric? 1:00:27 Credits and Outro This is Heliox: Where Evidence Meets Empathy Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter. Breathe Easy, we go deep and lightly surface the big ideas. Support the show Disclosure: This podcast uses AI-generated synthetic voices for a material portion of the audio content, in line with Apple Podcasts guidelines. We make rigorous science accessible, accurate, and unforgettable. Produced by Michelle Bruecker and Scott Bleackley, it features reviews of emerging research and ideas from leading thinkers, curated under our creative direction with AI assistance for voice, imagery, and composition. Systemic voices and illustrative images of people are representative tools, not depictions of specific individuals. We dive deep into peer-reviewed research, pre-prints, and major scientific works—then bring them to life through the stories of the researchers themselves. Complex ideas become clear. Obscure discoveries become conversation starters. And you walk away understanding not just what scientists discovered, but why it matters and how they got there. Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter. Breathe Easy, we go deep and lightly surface the big ideas. Spoken word, short and sweet, with rhythm and a catchy beat. http://tinyurl.com/stonefolksongs
- Transcript
- Chapters
