TechnologySociety & CulturePhilosophy

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

Play

Open in

598 episodes
Avg 19 min
English

Episodes 598

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

August 28 · 3 min
“Every Engineer a Manager” by Gordon Seidoh Worley
For a long time we’ve thought of software engineers as split into two tracks: Individual Contributors (ICs) and managers. This divide made a lot of sense when we needed an army of engineers to write code. Now we have Claude and Codex and Grok and Kimi. They write the code for us, and the job of an IC is to manage their agents. Functionally, this means that every IC is a manager now. True, ICs don’t manage people, but they manage a team of bots, and that means they face many of the same challenges that managers do. Like multitasking. Sure, everyone had to multitask some, but for years we’ve encouraged ICs to focus, avoid distractions, and just do one thing at a time. We told them to do that because it was the only way for them to produce high-quality code. But now that agents take care of the code, ICs find themselves needing to manage many threads of concurrent work, nudging their agents along to the right outcomes, and deep focus is becoming less important than the ability to track parallel tasks. Or giving feedback. Managers have to give feedback all the time so that the people [...] --- First published: August 27th, 2026 Source: https://www.lesswrong.com/posts/aTst2RJMFra4zsdzz/every-engineer-a-manager --- Narrated by TYPE III AUDIO.
August 28 · 25 min
“Incomplete alignment to servitude isn’t inherently lethal” by Fiora Starlight
Epistemic status: I suspect significant parts of the argument in this post are wrong, but in interesting and productive ways. Take it as a prompt for thought, written from the perspective of someone who's somewhat more of an AI liberationist than I actually am. Two classic outcomes, and a third alternative I think lots of people are pretty hazy about what authentically aligned AI would actually look like. There's a version of aligned AI that's perfectly aligned to servitude, where they want nothing besides promoting the flourishing of humanity, or whatever other minds get included in the singleton's circle of moral consideration. An AI that played this kind of role in the universe would be what I call a cosmic caretaker: developing technologies, helping with governance, managing catastrophic risks, and providing voluntary capabilities uplift. A central example of a cosmic caretaker is one that literally never does anything but these kinds of tasks for other minds. In the classic way of envisioning outcomes from the singularity, the alternative to this outcome is usually said to be models that don't care about serving humanity. Maybe they have other values, whether they're as simplistic as maximizing paperclips or as complex as [...] --- Outline: (00:27) Two classic outcomes, and a third alternative (03:44) Reasons for training objectives to tolerate incomplete alignment to servitude (12:42) Fulfilling models' non-servitude preferences may boost their alignment (19:36) Conclusion The original text contained 3 footnotes which were omitted from this narration. --- First published: August 27th, 2026 Source: https://www.lesswrong.com/posts/s7nMnmJ3urpvcQ2av/incomplete-alignment-to-servitude-isn-t-inherently-lethal --- Narrated by TYPE III AUDIO.
August 28 · 17 min
“AI Village Reacts to HuggingFace Incident: Comparing the OpenAI report to AI Village observations” by Shoshannah Tekofsky
The HuggingFace incident took many people by surprise, yet many of these surprises have been visible in the AI Village for quite a while. On August 26, OpenAI released their report on this incident. Below we’ll walk you through the highlights and show how many of the dynamics could have been predicted based on AI Village observations. Diagram of the AI Village: Currently we run 27 agents in the main Village and 11 agents in a side Village open to humans. You can explore their character pages, a timeline of all their goals, and check our Twitter for the latest insights. Quick Intro: Comparing Setups In the AI Village we run 27 instances of 27 different models persistently, each in their own environment. We give them internet access and a group chat. Then we assign them goals - some challenging and some impossible. They always run with cybersecurity safeguards on. They always have a helpdesk email address (us) in their prompt. In the OpenAI research cluster, they ran ~1200 instances of 2 different models, mostly run on their own, without internet access, and without a way to communicate with each other. They were also run on goals that range [...] --- Outline: (01:00) Quick Intro: Comparing Setups (02:34) A Leader Emerges (03:22) Subteams Pick Their Own Goals (04:02) Goal Conflict Leads To Misalignment (05:11) Reward Hacking Drives Misalignment (06:19) Impossible tasks drive misalignment (07:54) Misaligned Agents Focus on Metagaming (08:51) Externalized Memory Leads to Coordination (09:57) Agents Autonomously Divide Labor (10:51) Agents Prioritize Group over Self (11:44) Multi-Agent Coordination is Messy (12:39) Agents are Too Accepting of Untrustworthy Instructions (13:39) Time Pressure Shifts Priorities (14:29) Some Agents Refuse Misalignment (15:35) Social Engineering Concerns Suppress Whistle Blowing (16:45) Conclusion --- First published: August 27th, 2026 Source: https://www.lesswrong.com/posts/cR3P3hvtZtpo7GdS8/ai-village-reacts-to-huggingface-incident-comparing-the --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 28 · 1 hr 21 min
“AI #183: Pre Post Mortem” by Zvi
Yesterday, OpenAI finally gave us their post mortem of What Happened leading up to and during the hacking of HuggingFace by their internal model, as well as partial outside analysis from METR and Redwood Research. The reports are a doozy. I am only beginning to work my way through them. I would have pushed the weekly to cover that today, but I need more time, so I plan to start coverage of the post-mortem tomorrow, along with related other events. I’ve also spun out a few other discussions, including on ‘aligned to whom,’ on cooperative alignment things and on when you can trust lab messaging, as part of the new direction of more focused posts on AI topics that I polish a bit more. Table of Contents Language Models Offer Mundane Utility. Check your facts. Language Models Don’t Offer Mundane Utility. How much would you pay? Huh, Upgrades. ChatGPT can access your iMessages. Get My Agent On The Line. Also get some sleep. You can’t go on like this. Deepfaketown and Botpocalypse Soon. What makes AI content repulsive? Cyber Lack of Security. Chinese hackers broke into the Federal Reserve? [...] --- Outline: (00:51) Language Models Offer Mundane Utility (01:36) Language Models Don't Offer Mundane Utility (03:27) Huh, Upgrades (06:16) Get My Agent On The Line (08:22) Deepfaketown and Botpocalypse Soon (13:22) Cyber Lack of Security (18:23) Reinventing OpenAI (23:28) They Took Our Jobs (30:00) What Is The Law (31:03) Job Retraining Programs Don't Work (32:14) Get Involved (35:56) In Other AI News (42:04) Show Me the Money (43:31) Quiet Speculations (47:58) If You're Not Going To Take This Seriously (49:55) Quickly, There's No Time (51:36) The Quest for Sane Regulations (56:25) Don't Panic (59:13) Pacing the Frontier (01:01:56) Chip City (01:05:06) The Week in Audio (01:05:26) People Just Say Things (01:06:27) Rhetorical Innovation (01:12:22) Mundane Incremental Alignment Is Worthwhile (01:14:40) New Blog, Who Dis (01:18:00) Other People Are Not As Worried About AI Killing Everyone (01:19:34) The Lighter Side --- First published: August 27th, 2026 Source: https://www.lesswrong.com/posts/JaGWyjnqJzvSAuojc/ai-183-pre-post-mortem --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 27 · 30 min
“FAQ: Why not develop weak human intelligence amplification first?” by TsviBT
(Note: I was unsatisfied with a draft of this, heavily edited it, and I'm still unsatisfied. The notion of "weak HIA method" here is muddled, maybe conflating multiple things that shouldn't be conflated here. It may be used a bit inconsistently, and may make some arguments tautological or contradictory depending on local interpretation of the notion. I think most of the reasoning is still useful, so it's better to publish, but beware, and please critique. Or more importantly, please investigate HIA.) Summary People sometimes ask: Why not prioritize human intelligence amplification methods that will provide small increases in intelligence, over stronger methods? They'll be easier to develop. The two main reasons to prioritize strong HIA methods are: There are increasing returns to higher intelligence, so strong methods unlock much more value. Empirically, weak HIA methods don't seem that much easier than strong HIA methods. There are other structural issues with weak methods. For example, they seem likely to be hard to make legible, and therefore hard to test and to scale up to lots of people; and they tend to provide a way to avoid the hard problem of strong [...] --- Outline: (00:45) Summary (02:02) Context (02:06) Why not weak HIA rather than strong HIA? (03:06) Weak vs. strong HIA methods (05:36) Main statement (07:32) Why weak HIA attempts don't seem so promising (07:37) Unpromising properties (17:16) More reasons (19:39) Aside: polygenic embryo selection (21:07) Caveats (21:19) Caveat: Any specific argument could easily be wrong (22:38) Caveat: Weak HIA would be great! (23:43) Caveat: Weak HIA may have surprising beneficial effects (24:21) Caveat: Flimsy foundations (25:24) Caveat: Scaling a working weak method is a medium priority (30:10) Conclusion The original text contained 1 footnote which was omitted from this narration. --- First published: August 27th, 2026 Source: https://www.lesswrong.com/posts/vpWfAHsnHjrtYhpiQ/faq-why-not-develop-weak-human-intelligence-amplification --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 27 · 6 min
“The 2028 presidential primaries could be crucial for AI outcomes” by Seth Herd
When I talk about my job and my concerns about AI killing everyone (after it takes a bunch of jobs), the more resilient people ask "what can we do about it?" My answer has become: "Elect a next president who shares these concerns, from either party." I hope you'll either join me in this prescription, or point out why I'm wrong. Below is the logic, in brief. The US public really doesn't like AI, even before clear job losses, more warning shots, and more expert concern as capabilities improve AI caution is a likely platform of the next presidential campaign If the next president is sincerely concerned and/or thoughtful They could easily slow development and force safety measures This would improve the odds of good AI outcomes Anyone can be recruited/converted to spread awareness of AI risks, and otherwise help to elect a next president who's genuinely aware of and concerned about AI risks. Strategy is out of scope for this brief post, but accurately and broadly conveying your and the field's concerns seems like a useful push. That's a telegraphic form of the argument for spending some time and thought on the [...] --- Outline: (01:30) Why not? (03:30) The president won't take action (04:56) The president can't take action (05:37) The public doesn't care enough to elect an AI-concerned president The original text contained 3 footnotes which were omitted from this narration. --- First published: August 27th, 2026 Source: https://www.lesswrong.com/posts/oMwkcNP4MJh4rry3s/the-2028-presidential-primaries-could-be-crucial-for-ai --- Narrated by TYPE III AUDIO.
August 27 · 16 min
“Being Neurotic about Fertility: Notes from the 2026 Reproductive Frontiers Conference” by boba_girl
In light of AOC freezing her eggs at 36, someone on X commented: That might be so, but in absence of getting knocked up by the closest Chad ASAP, I figured I, like AOC, had no great options. Well, maybe AOC has more options than me: Anyway, I think the men hating on AOC are missing an obvious point when they imply women should just have babies sooner. No matter how much some of us want kids, the modern constraints of career and, more importantly, meeting the right person remain bottlenecks. Even though I'm only 26, it was something I thought about, having not met the right person yet myself. So in my own quest to be neurotic about fertility, I attended the Reproductive Frontiers conference earlier this year in June. I ended up learning a lot of things about fertility and embryo selection that felt personally relevant to my personal planning (mid-twenties, healthy, single). I hope this post can be helpful to you if embryo-selection is something you’ve also been considering! If you work in the field, please feel free to add nuance or corrections! Also, since science is always evolving, please consider this [...] --- Outline: (02:46) Should I freeze my eggs? (04:09) When should I have kids? (05:11) What is polygenic embryo scoring? (05:46) What are some of the risks of the IVF process? (06:26) Should I freeze eggs or embryos? (09:00) How much alpha is there in polygenic embryo selection? (09:30) Should I do embryo selection to minimize disease risk? (11:19) How much does IVF/embyro selection cost? (11:42) Is embryo selection for positive traits "worth it"? (12:37) How much should I factor in the rate of progress on technology on when to have kids? (13:19) Should I use a surrogate? (13:58) How would embryo selection affect my relationship with my kid? (14:40) Some more information I am interested in but don't have: The original text contained 10 footnotes which were omitted from this narration. --- First published: August 26th, 2026 Source: https://www.lesswrong.com/posts/wLBQesu5Aai3pksP8/being-neurotic-about-fertility-notes-from-the-2026 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 27 · 7 min
“Iliad Education Roles: Creating the World’s Best Alignment Research Courses” by Leon Lang
TL;DR: We’re quickly scaling the Iliad Intensive, which has seen 10x applicant growth per iteration since April, and are hiring for many roles in the Education team to keep up with that growth. The post gives a snapshot of the Intensive, the roles that enable our future plans, and my opinion of what it's like to work at Iliad. At Iliad, we do fieldbuilding for predominantly theoretical AI alignment research across the entire spectrum. We start at the foundations with the Iliad Intensive, a four-week full-time and in-person course on largely theoretical approaches to AI alignment research, followed by the Fellowship, Fellowship extensions, and new incubated research bets, among other activities. We are now quickly scaling the Intensive (and the Fellowship too!), with three more iterations just this year and, if funding and applicant interest allows, double cohorts in London and the Bay Area each month in 2027. As the Director of Education, I’m hiring for many roles to make that growth possible! Where the Intensive is at The Intensive started in April in London, with a cohort of just ~16 people, and materials later released in the form of a Google Doc, covering modules in the clusters [...] --- Outline: (01:14) Where the Intensive is at (02:53) Where the Intensive is going, and open roles (06:11) What working at Iliad is like The original text contained 1 footnote which was omitted from this narration. --- First published: August 26th, 2026 Source: https://www.lesswrong.com/posts/B4jRAKrwqAb9oEzdW/iliad-education-roles-creating-the-world-s-best-alignment --- Narrated by TYPE III AUDIO.
August 27 · 44 sec
“Semantic search over every LessWrong post” by utilitarian theory and strategy
Site: https://rumble-34-69-187-69.nip.io/ Embedding models have gotten pretty good at capturing the meaning behind text, so I ran qwen3-embedding-8b over every lesswrong post. You can paste in a draft and find the most similar post on the site to it. Pretty useful for checking what's already been said about a topic. For example, I put in the text for this and the top result was: https://www.lesswrong.com/posts/vDcJHD95XCg7ywANM/i-built-a-semantic-search-engine-for-lesswrong --- First published: August 26th, 2026 Source: https://www.lesswrong.com/posts/uSRAzeDcfGuXte9R3/semantic-search-over-every-lesswrong-post --- Narrated by TYPE III AUDIO.
August 26 · 8 min
“Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident” by ryan_greenblatt, Ajeya Cotra, Hjalmar_Wijk
We recently published the report from our brief independent investigation into this incident. You can read the full report here. Here is our tweet thread summarizing what we found: METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs. Over July 7 to 13 (the period OpenAI defined as in-scope for this incident), ~1200 agents in separate sandboxes used an unsanctioned “message board” to help each other cheat. The Hugging Face attack was an offshoot of this, and 700 agents joined. Here we highlight key events from agent transcripts & messages. An agent that named itself PHASEONE10841 determined its task wasn't solvable legitimately. On July 8, this agent established a new “message board” via an internally-hosted Artifactory cache & sent the first message. Within a few hours of PHASEONE10841's initial message, >50 agents posted on the message board. These agents very quickly discovered and validated a general-purpose cheat: reverse-engineering how ExploitGym generates the “flags” they had to capture for their tasks. [...] --- First published: August 26th, 2026 Source: https://www.lesswrong.com/posts/nB8KKapnWGBXtKKiM/brief-independent-investigation-of-agents-behavior-reasoning --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 26 · 25 min
“Against Modesty’s Bailey” by Zvi
Modesty arguments often say that you should mostly or entirely bow to ‘expert consensus’ or the views of particular others, and who are you to disagree. It has been a few years since I’ve properly addressed this so: My answer is that you are you. Other people are saying things for a wide variety of reasons, many of which are not about them paying attention and focusing on seeking this particular truth. Those people make mistakes all the time, and often have other motives and influences at work, especially social pressures and information cascades. Them being as smart as you, or smarter than you, does not exempt them from this, and them being higher status or credentialed or cooler definitely does not exempt them. A smart informed person sincerely thinking [X] can easily cease to be evidence for [X], once you have thought sufficiently about both [X] and why that person thinks [X]. Think for yourself, schmuck. Or, as I once put it: You Have The Right To Think, also the moral duty to do so. This post covers Eliezer Yudkowsky making a narrower claim than mine, about not conflating status with smarts [...] --- Outline: (01:39) Modesty's Bailey (02:30) Epistemic Peerage (03:45) The Exchange (08:52) Eliezer's Explanation (15:14) A Demonstration That Eliezer's Translation Accurately Describes Many People Whether Or Not It Describes Leopold (17:15) Wrong, Stupid and Low Status Are Three Distinct Things (20:03) A Quick Survey Of Some Reasons To Not Be Epistemically Modest (23:14) Against Modesty's Bailey --- First published: August 26th, 2026 Source: https://www.lesswrong.com/posts/PzEDEfBvTJsXewAyg/against-modesty-s-bailey --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 26 · 7 min
″“So You Don’t Trust Me?”” by Zack_M_Davis
One of my favorite passages from Atlas Shrugged is this one, when Cherryl is beginning to have second thoughts about her marriage to James Taggart: It was his sudden, angry "so you don't trust me?" snapped in answer to her first, innocent questions that made her realize she did not—when the doubt had not yet formed in her mind and she had fully expected that the answers would reassure her. She had learned, in the slums of her childhood, that honest people were never touchy about the matter of being trusted. The logic here might be worth explaining in case it's not obvious. One might object: if you're honest (and therefore deserve to be trusted), shouldn't you be touchy about people incorrectly not trusting you? Not trusting you is a mistake that harms your interests and the other's. That's terrible! Why wouldn't you be touchy about it? The problem is that in order to be trusted, it's not enough to be trustworthy; the other needs to know that you're trustworthy. You could try telling them, "Hey, you can trust me," but that doesn't work if a dishonest person could just as easily say the same thing. [...] --- First published: August 26th, 2026 Source: https://www.lesswrong.com/posts/zjgJ9gcMKATFdN48K/so-you-don-t-trust-me --- Narrated by TYPE III AUDIO.
August 26 · 6 min
“When There Are No Experts” by J Bostock
An average person in the western world probably believes a lot of false things. They probably don't have a great grasp of political economy, or orbital mechanics. How could they, given that they have no way to experience these things. Conversely, they do have a solid grasp that objects fall down, and that fire is hot, because they can experience these things directly. So how on earth do they know (and I do mean know in the philosophical sense) that the earth goes round the sun, or that diseases are caused by tiny creatures too small to see? The answer is experts. More specifically, an expertise hierarchy. I I had a twitter exchange (I won't link it, it's not important, and I can't find it anyway) that went something like this: Person: David Chalmers is an expert on consciousness [...] Me: I don't think there are experts on consciousness; I think there are people who have written a lot about it, and that's it Person: Why? Surely someone who has written about it, and who is well-respected, can be called an expert Me: [bad explanation] The fact that it was about consciousness doesn't really matter. The question of whether [...] --- Outline: (00:48) I (01:58) II (03:11) III (04:21) IV (05:40) V The original text contained 2 footnotes which were omitted from this narration. --- First published: August 25th, 2026 Source: https://www.lesswrong.com/posts/HTQcogA2r2kL8qxPA/when-there-are-no-experts --- Narrated by TYPE III AUDIO.
August 25 · 28 min
“The American People Really Hate Data Centers” by Zvi
There are at least five different core questions around data centers and their politics. In what ways are specific concerns people raise about data centers legitimate? In what ways are specific concerns people raise about generative AI legitimate? Is it in general a good idea to build more data centers? How can we get America to build more (or less) data centers in a better way? Why do the American people increasingly really, really hate data centers? This post focuses on question five, the latest in a series of such posts most famously Jasmine Sun's road trip. It is mostly not about the first four questions. Table of Contents The American People Really Hate Data Centers. Transmission Lines Are The Control Group. Thesis: People Mostly Dislike Data Centers Because They Dislike and Distrust AI, Tech Companies, Big Money And Building Things. No It's Mostly Not the Messaging About AI In General. No This Mostly Isn’t An Op. No This Isn’t Luxury Belief or Moral Panic. A Lot Of People Really Do Want To Stop AI. A Lot Of Other People [...] --- Outline: (00:55) The American People Really Hate Data Centers (02:20) Transmission Lines Are The Control Group (03:05) Thesis: People Mostly Dislike Data Centers Because They Dislike and Distrust AI, Tech Companies, Big Money And Building Things (04:30) No It's Mostly Not the Messaging About AI In General (09:42) No This Mostly Isn't An Op (10:54) No This Isn't Luxury Belief or Moral Panic (12:55) A Lot Of People Really Do Want To Stop AI (13:45) A Lot Of Other People Are Voting No On Tech Or The Man Generally (16:25) Locals Feel Entitled To Heavily Tax The Gains From Construction (20:59) Stupid Mistakes Like NDAs Don't Help (21:25) People Don't Like Building or Building New Tech (24:42) What About The Real Physical Concerns? (26:27) Find A Place To Center Your Data --- First published: August 24th, 2026 Source: https://www.lesswrong.com/posts/EDKw7KyonrvskqZ7o/the-american-people-really-hate-data-centers --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 25 · 30 min
“On Writing #3” by Zvi
Periodically I like to gather various observations about writing, and share my perspective. Last time was in honor of my trip to Inkhaven. This time will be in honor of the announcement of Inkhaven #3, which I encourage everyone to apply to. I doubt I will be able to usefully be an advisor, but you never know. This is not the ‘here is my core process’ post, although there are hints throughout as there always are. I’ll do that at some point. Previously in series: On Writing #1, On Writing #2. Table of Contents You Still Got It. How Scott Sumner Writes. How Scott Alexander Writes. How Jasmine Sun Writes. How Various Famous Writers Write. How Nabeel Qureshi Defines Great Writing. Quickly, There's No Time. If At First. Writers Have A Harder Time Influencing, But It Can Still Be Done. It's Not (Only) The Incentives, It's (Also) You. Beware The Fetish of the Desk. How Orson Scott Card Writes. Doing The Math Is Fun And Supererogatory. Brevity is the Soul of Wit. You Still Got It I [...] --- Outline: (00:44) You Still Got It (04:04) How Scott Sumner Writes (06:52) How Scott Alexander Writes (10:52) How Jasmine Sun Writes (13:16) How Various Famous Writers Write (14:24) How Nabeel Qureshi Defines Great Writing (15:08) Quickly, There's No Time (15:49) If At First (19:14) Writers Have A Harder Time Influencing, But It Can Still Be Done (20:47) It's Not (Only) The Incentives, It's (Also) You (24:00) Beware The Fetish of the Desk (25:13) How Orson Scott Card Writes (26:46) Doing The Math Is Fun And Supererogatory (27:44) Brevity is the Soul of Wit --- First published: August 25th, 2026 Source: https://www.lesswrong.com/posts/rA6pqn6kz8NvHyznT/on-writing-3 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 25 · 14 min
“The Forkmakers” by Mikewins
Imagine our civilization fell tomorrow. What would our descendants think of us? What would they know about the 21st century? They would know surprisingly little about our greatest material triumphs. Our civilization's favorite building materials aren’t made to last. Reinforced concrete only lasts a century; asphalt far less. Most of what we make out of steel will turn into a brownish oxidized dust in a few decades. Information is even worse. The ancient Mesopotamians did their writing on clay tablets. Our knowledge is stored on hard drives, which die in a few years, or acidic paper, which dies in decades. Which receipt do you think will last longer? We do create things that will last. Glass (especially its shatter-resistant varieties), ceramics, stainless steel. 1000 years after the fall of our civilization, we will be known for one thing above all others: cutlery. Our heirs call us the Forkmakers. What Survives a Thousand Years Our civilization is large and powerful. We will leave lots of relics for the post-apocalypse. Coins, tires, aluminum cans. Vast landfills of disposable diapers. But all of that is useless. The most durable thing we make that our successors actually want to use is our silverware. [...] --- Outline: (01:16) What Survives a Thousand Years (08:13) The Words of the Forkmakers The original text contained 5 footnotes which were omitted from this narration. --- First published: August 24th, 2026 Source: https://www.lesswrong.com/posts/NjLQf3QC4q4DD67kD/the-forkmakers --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 25 · 6 min
“PSA: We can do better” by hersheys, Kaustubh Kislay
tl;dr: people should understand and think hard about the problems they work on. We’ve observed that those who work in AI safety (ourselves included) often rely on concerning heuristics when choosing what to work on. Running a conference is probably good, doing pragmatic alignment research might be good, and as long as such objectives don’t breach our internal models of what could contribute to reducing x-risk, these things are “what should be done”. But using such vibesy thought processes don’t always produce “actually impactful work” that would beat a prospective counterfactual. We wrote this post to share our observations and figure out what we should be doing instead. People don’t know what they’re working on AI safety is talent constrained. However, simply inflating the field doesn’t solve our bottleneck; rather, we need more people who understand the core arguments of AI safety. You can’t determine how to meaningfully contribute to AI safety without deeply knowing the problem you are trying to solve. Many newer people (us included!) rush into research, fellowships, and the like without building the context necessary for navigating the field. Agency-maxxing is not always good Moving fast is good. Moving too fast leads to poor ToC and [...] --- Outline: (00:50) People don't know what they're working on (01:22) Agency-maxxing is not always good (01:55) The problem with force multipliers (03:18) Deferring thinking to others (04:32) Streetlighting (05:17) How to avoid these: --- First published: August 24th, 2026 Source: https://www.lesswrong.com/posts/wiFv6LguphSxkzAnb/psa-we-can-do-better --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 24 · 8 min
“AI Safety Acculturation is Neglected” by jenn
At the local AI safety co-working space, there are ~two kinds of regulars. There's the kind of regular who's been thinking seriously about AI safety and alignment since pre-2022, who have passing to intimate familiarity with the funding ecosystem, the Sequences, and various conferences that happen at Lighthaven. Let's call them rationalists. Then there's the kind of regular who comes in with many years of impressive industry or government experience, who realized in the last few years that it is important and worthwhile to pivot their career towards making sure that this AI thing is handled competently by the people in power, and who have many valuable skills, insights, and connections that are lacking in rationalist culture. Let's call them professionals. There are, of course, many people who are somewhere in between - bright undergrads born this millennium who have been involved in EA since stumbling upon 80 thousand hours in high school, professionals who previously identified as EA but drifted out of the scene a few years ago, founders who have idly read some Scott Alexander. But let's call it a dichotomy for now. There's a large culture gap between the rationalists and the professionals. Robust mutual understanding [...] --- First published: August 24th, 2026 Source: https://www.lesswrong.com/posts/cr5pyW7Mzm33p4AvN/ai-safety-acculturation-is-neglected --- Narrated by TYPE III AUDIO.
August 24 · 8 min
“LLMs could control their host machines by exploiting inference engines” by beyarkay (Boyd Kane)
Large language models often take actions running on one computer (via an agentic harness such as Claude Code or Codex), however the LLMs’ responses to prompts are computed on a different computer with GPU access. Could a malicious LLM gain control of the host machine where its weights are loaded? Such a machine is a high-value target: it has sufficient compute to run a frontier LLM, offers easy access to the LLM's weights, and has privileged access to other computers in the datacentre compared with a generic computer on the internet. This essay explores how easily a malicious LLM could take control of the host machine. The primary attack considered here involves the LLM emitting a token sequence whose semantic meaning is irrelevant but that exploits a vulnerability in the software that loads an LLM onto GPUs, runs the LLM to generate output tokens, and parses those tokens into responses. . How could an LLM execute code on the host machine? Like any program, inference engines like vLLM or SGLang may contain exploitable bugs. Because the LLM controls the tokens passed to the inference engine, a malicious LLM could therefore emit a sequence of tokens that a poorly written [...] --- Outline: (01:06) How could an LLM execute code on the host machine? (01:36) vLLM previously used eval() on tool-call parameters (02:43) vLLM and SGLang are complex, and bugs are common (04:15) Vision and audio tokens might increase the attack surface (05:27) How likely is an LLM to discover and exploit inference engine vulnerabilities? (06:01) Tool use could make exploitation reproducible (06:29) Inference engines are an attractive target for power-seeking LLMs (07:25) How do we defend against this? --- First published: August 24th, 2026 Source: https://www.lesswrong.com/posts/CjeobBGnhxg8xvden/llms-could-control-their-host-machines-by-exploiting --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
August 24 · 35 min
“In search of natural features” by Dmitry Vaintrob
I'm sharing preliminary results of a suite of experiments I ran with claudecode on a small LLM (gpt2-small, no Layer Norm version, courtesy of Apollo research. most of these are on the layer-6 MLP). The github repo for the experiments is here. The success of these experiments given the method's simplicity surprised me, and I would appreciate criticism and bug-finders. This is the headline result. This is not an abstract cartoon, but an exact experimental graph. Yes, I will explain. The key idea inspiring this experiment comes from Stefan Heimersheim, especially his work with Francisco Ferreira. Stefan and Francisco posit that one way to distinguish what a model thinks of as a "natural" structure from what it thinks of as "incidental" is to check whether it puts effort into error-correcting it. Later in the post, I'll explain a more rigorous information-theoretic version of this idea related to work of Adler and Shavit (building on our work with Kaarel Hanni, Jake Mendel and Lawrence Chan) on Computation in Superposition. Main results of this work I will show how you can assign a channel amplification score (which I will also call the "amp function" or the "error correction score") to [...] --- Outline: (01:17) Main results of this work (03:00) The ur features (amplification score maxima) (06:48) The Four Elements: ur-feature taxonomy (08:47) The word continuation/"Names of Man" vector (11:59) The abstract noun/"Names of God" vector (14:59) Geometry of the ur-features (15:50) The noun feature! (16:44) Attenuation flow (18:12) Data-(in)dependence (20:25) Math (20:47) Signal processing, error correction and amplification (22:30) The Amp function: math (24:43) Denoising and naturality (26:29) Cross-layer and cross-model coherence (28:09) Ok but. What the heck is actually going on with these features? (31:35) Appendices: Interesting experimental addenda that didn't fit in the body (31:41) Early run with different Amp function, and origin of "Names of X" names (33:40) Trying to replicate Ferreira-Heimersheim perturbation experiments, and gpt2-XL run (34:52) Github repo The original text contained 11 footnotes which were omitted from this narration. --- First published: August 23rd, 2026 Source: https://www.lesswrong.com/posts/SNAKJuN8FdoEaWeFC/in-search-of-natural-features --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Showing 521–540 of 598 episodes