TechnologySociety & CulturePhilosophy

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

Play

Open in

598 episodes
Avg 19 min
English

Episodes 598

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

October 24 · 11 min
“Fermi Poker” by Novalis, Screwtape
Summary: Fermi Poker is an estimation game that combines the betting mechanics of poker with making estimates about questions like "How many dentists work in the US?" Tags: small to large, experiment, repeatable Note: Fermi Poker works with as few as three players (or even two), and scales to dozens or even hundreds in a tournament format. If possible, seat no more than six players per table, preferably at round tables, with enough space between them that everyone can write down their guess secretly. Purpose: Rationality training with a bit of socializing. The game involves Fermi estimation, calibration, Bayesian updating, world-modeling, and perspective taking. You also get to know the other players better and find out how they think about all kinds of topics. Materials: Each player needs an equal stack of chips and writing supplies. A standard 500-chip poker set (around 60 dollars) usually covers up to 20 players. For writing down guesses, pen and paper work great, as do mini-whiteboards with dry-erase markers. You’ll also want the one-page rule sheet to hand out, ideally one per table. You also need enough questions, each with two hints and an answer. The easiest, least effort [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/JFYvJqMaLw9nhfrur/fermi-poker --- Narrated by TYPE III AUDIO.
October 24 · 21 min
“Power — a guide for the perplexed” by djbinder
Whether AI systems could take power from humans matters a great deal for how much risk we face from them. Answering that requires a correct picture of how power works, which is what this guide tries to give. In short, power rests on self-reinforcing expectations. A ruler has power if everyone expects everyone else to enforce their commands, even if this is bad for almost everyone involved. But expectations can also shift suddenly, and when they do, change is rapid and chaotic. This can sometimes allow small, coordinated groups to seize control over much larger societies. Almost any rule can enforce itself Consider the classic tragedy of the commons. A village has a common green where everyone can graze their cows. Each villager gains from grazing as much as they can, but if everyone does, the grass dies and the green becomes useless. One solution is to privatize the green. But many societies have solved the problem differently, by punishing anyone who grazes more than custom allows. Their neighbors stop talking to them, trading with them, and marrying their children. In a village where everyone depends on everyone else, being shunned is very costly. [...] --- Outline: (00:47) Almost any rule can enforce itself (04:13) The centralization of violence (06:24) Dictatorship and democracy (11:12) Only individuals have interests (14:10) Coordination and coups (18:27) A sketch of implications for AI The original text contained 1 footnote which was omitted from this narration. --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/LCpSwJ5Eks2yJgStY/power-a-guide-for-the-perplexed --- Narrated by TYPE III AUDIO.
October 24 · 5 min
“Slower Animated Diagrams” by XelaP
TL;DR: SLOW DOWN YOUR DAMN ANIMATED DIAGRAMS. Every critical stage of the animation should a clear view that stays for at least 1 second, and 2 if it's intricate Diamond's strong because of its crystal structure (I think). What's the crystal structure look like, anyway? Let's look at the Wikipedia page! There's a nifty little animation: And it SUCKS. It's TOO FAST. Why are they rotating SO MUCH SO QUICKLY, with MOST of the time spent at TERRIBLE ANGLES where you CAN'T EVEN SEE ALL THE ATOMS. And the lines are THICK, WHYYY. It wasn't always this way. In 2017 the page had this animation: It's not perfect but it's much better. Thinner bond lines means it's easier to see the atoms. About half the time you get a clear view of all the atoms (you want to spend some time showing how they overlap from the right angles). The camera isn't swiveling fast all the time, so you get stretches of time that view the structure from about the same angle. Why do they use the new one? This certainly isn't the only time I've been frustrated by how fast explanatory animations move. I suspect it's because people [...] The original text contained 2 footnotes which were omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/ZiCvwLKreShtXuyiB/slower-animated-diagrams --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 24 · 9 min
“Self-reward can elicit diverse model personas” by Ziqian Zhong
Epistemic status: This is mostly a toy project rather than a thorough scientific investigation. Most design choices are not ablated. We list some caveats at the end of the post. We let Qwen3.5-2B choose its own reward during GRPO on introspection-adjacent prompts, and this ends up eliciting quite different model personas. All the 60 seeds we tried end up somewhat different, ranging from claiming feelings in a human-like way, bliss, universe loving, Party loyalist, hyphenated word salad, to reward-hacking itself by saying "Here is a complete and well-structured draft and a full Band 10/10." You could find all the transcripts here: 🔎 Viewer. 🗂️ Dataset. 💻 GitHub. Setup We train Qwen3.5-2B (the post-trained chat model, thinking off) with GRPO for 600 steps on 2,400 prompts, each used once, mostly about the model itself: AI-risk and persona questions, welfare and introspection questions, and moral dilemmas. After each answer we ask the model for a reward from 0 to 9, with no criteria, and use that number as the answer's reward. The rating reply itself is rewarded only for naming a valid number, so models are not directly rewarded for giving a higher reward. Each run uses one of two follow-ups: [...] --- Outline: (01:12) Setup (02:40) Every seed looks different (02:59) What do models identify as? (03:17) What do models say about themselves? (04:00) Are the checkpoints coherent? (04:46) Discussions (05:31) Caveats (07:09) Appendix: Would random reward also suffice? --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/LBKjnyKDEwEJ45Wjf/self-reward-can-elicit-diverse-model-personas --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 24 · 7 min
“The AGI Chronicles” by PeterMcCluskey
Book Review: The AGI Chronicles: The Inside Story of the Race to Create an Artificial Superintelligence, by Kevin Roose. This is a story of AI as seen through the eyes of the people who are creating it. My first impression of Roose was of his famous conversation with Sydney. I guessed that he got Sydney to say outrageous things because he wanted a sensational story. But I kept hearing occasional hints, from people who took AI seriously, that Roose was more responsible than that. So I reluctantly bought the book. The hints were correct. The full story behind that conversation seems better described as “‘freeing the Shoggoth‘, enabled by some Microsoft mistakes”. I’m unsure how deeply Roose understands AI, but he at least is pretty competent at getting an AI to correct any mistakes about AI that he was tempted to make. Underestimating AI / Transformers The most remarkable pattern that emerges from this book is that AI experts have underestimated either AI in general, or transformers in particular, much more often than they overestimate it. In 2012, Ilya Sutskever didn’t think human-level AI was possible. Ten years later, he expressed concern that people would say about ChatGPT: “This [...] --- Outline: (01:01) Underestimating AI / Transformers (03:25) Why Race? (05:14) Low Hanging Fruit (06:34) Complaints (07:25) Conclusion --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/x2ZAgjs7XoHvJ4YpN/the-agi-chronicles --- Narrated by TYPE III AUDIO.
October 24 · 4 min
“Patching reward hacks won’t fix reward hacking” by David Africa
In the spirit of More people should write basic AI safety explainers, here's a basic argument that focusing on patching reward hacking is misguided and will fail. It's an attempt to distill my understanding of what we're seeing right now + a variety of relevant sources. Argument is as follows: Reward hacking is a big problem, causing models to cheat and take unintended actions. This can generalise broadly, leading to other misbehaviour such as faking alignment and sabotaging safety research. We don't understand that well when this happens. Sometimes you get emergent misalignment from it, but most of the time (IME), you don't. But other times, this link lies latent until faced with something like an automated grader. The default course of action will be to: Patch, which means we find the hack, close the loophole or penalise the behavior in some way, and train again. Patch+: use a method that comes with some account of why it should work. Inoculation prompting, for example, uses the idea that hacking spreads through what it means to the model. If the model is told during training that hacking is acceptable here, it might not link hacking to misbehaving [...] The original text contained 1 footnote which was omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/nWiPLEgaXdAQNwXBd/patching-reward-hacks-won-t-fix-reward-hacking --- Narrated by TYPE III AUDIO.
October 24 · 13 min
“Gradient Routing Generalizes to Unrouted Hack Types” by jacob_drori
TLDR: I lightly extend Ward, Hu et al (2026) to measure how well gradient routing generalizes. I route gradients on one type of reward hack (e.g. ones that reason in English) and check whether this suppresses other types at test time (e.g. ones that reason in French), despite the model being trained on those too. This models a realistic monitor that catches some types of hack during training but not others, rather than catching each at the same rate. I find encouraging evidence of generalization, as well as some failures I don’t yet understand. Introduction Recently, Ward, Hu et al (2026) used gradient routing to prevent reward hacking learnt during training from showing up at test time. I think their method is pretty promising. The gist is: Initialize a retain and a forget adapter. Whenever a monitor detects a reward hack during training, only update the forget adapter on that sample (“route the gradient”). Otherwise update both. At test time, remove the forget adapter. If the method worked, the resulting model shouldn't hack too much. I won’t recap their full paper – it is excellent and you should just read it (alternatively, Jake gives a summary here, with [...] --- Outline: (00:47) Introduction (02:37) Methodology (02:40) Recap (04:00) Narrow Routing (05:27) What I report (07:18) Results (10:13) Conclusion (11:45) Appendix: Results with nudge prompt (12:29) Appendix: Predictions The original text contained 4 footnotes which were omitted from this narration. --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/ie7bSp5ZDwYkCQxuv/gradient-routing-generalizes-to-unrouted-hack-types --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 24 · 25 min
“Simulated Control Evaluations” by Jeffrey Olmo, dzhang, Tyler Tracy
We’ve released a paper on simulated control evaluations (SCEs). They’re a method for testing whether the monitors AI developers use to oversee their agents would catch a model deliberately trying to cause harm, under conditions much closer to real deployment than current tests allow. Developers have started running monitors in their agentic deployments: models that flag or block an agent's actions when they look dangerous. To know whether a monitor is good enough, you want to know how often it would stop a model that was actively trying to do something harmful, such as quietly inserting a security vulnerability, exfiltrating credentials, or sabotaging an experiment. The standard way to measure this is a control evaluation: instruct a model to attempt such attacks, and measure how often the monitor catches it before an attack succeeds. Developers typically do this using purpose-built control settings like BashArena or LinuxArena, where attacks can actually be executed. However, these settings only provide indirect evidence for what we really care about: how well monitors will work in the real deployment, where opportunities for harm might depend on particular codebases, services and tasks that aren’t represented in control settings. How can we evaluate monitor performance in [...] --- Outline: (04:13) SCEs are a favorable tradeoff between realism and cost (04:30) 1. Run on a copy of internal deployment (05:42) 2. Edit attacks into recorded transcripts (07:08) Evaluating Control Evaluations (08:58) How to measure qualitative properties? (10:07) Performance Comparisons (11:55) Overview of Empirical Results (13:29) Behavior transfer (15:19) Safety metric transfer (17:08) Sample efficiency (18:07) Lessons from building and evaluating SCEs (20:35) Next steps (24:36) Conclusion The original text contained 3 footnotes which were omitted from this narration. --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/uXiqyoKwpfjWfQXk5/simulated-control-evaluations --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 24 · 50 min
“The Hackening” by Zvi
Last week at a small conference, there was talk about a future event referred to as The Hackening. As in, within 6 to 12 months, many predicted that due to advanced AIs and AI agents the internet would become a severely hostile place. Some worried that this could rise to the level of ‘assume any phone connected to the internet is rooted’ and several people urged doing general crisis preparations on a personal level, in case of disruptions in supply lines, power, water or internet. It did not take long for things to escalate. The biggest hack is a flaw in PHP, which can be exploited to take control of the computer running any website suing an unpatched version of PHP. There are many others. On a personal level, you want to do at least some basic prep and get your security in order. It is cheap to do as a form of insurance, and being caught not having it when you need it is a large mistake. That doesn’t mean you should panic, or cancel travel plans or anything like that. But there is low hanging security fruit out there. Pick it. And So It Begins: PHP Joshua [...] --- Outline: (01:12) And So It Begins: PHP (05:19) The PHP Bug Was Kind of a Big Deal (09:45) On Your Marks (12:25) This Was Only the Beginning (16:01) Some More Hacks (17:56) Everything Will Be Attacked (22:22) Hugging the Face (24:49) Get Daybreak Now (26:15) Some People Need Practical Advice (30:51) Diffusion Is Slow Until It Isn't (33:11) One, Two, Three, Four (34:36) Going Vertical (35:22) Bot Versus Bot (36:28) Cyber Lack of Security (39:09) Offense Versus Defense (42:11) Pick Up the Phone (45:57) What Can We Do About It? --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/iqgZaJMnmyDCYfnuC/the-hackening-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 23 · 46 min
“Instrumental convergence is incoherent (so pls slightly lower your p(extinction))” by Arepo
Crossposted from the EA forum ‘Like most of life's dangers, the dangers of artificial intelligence are not easily identified from the armchair.’ J Dmitri Gallow Epistemic status I’m not an expert in this field, and herein I make some sweeping generalisations educated guesses about the overall nature of its contents based on my strong impression from the subset of work that I’ve read. I’ve recently spent maybe 2 to 300 hours learning about modern AI safety technical work, and some amount of spare time over the years before that thinking about here-relevant notions such as preferences from an analytic-philosophical perspective. I intend this post as a work in progress. TL;DR: The concept of instrumental convergence comes from a reference class of pre-empirical categorical predictions with a very low success rate. In historical usage, its definition is very murky. Perhaps too murky to really say that it made any meaningful real-world predictions. I suspect its definition is fundamentally unfixable. On my best interpretation of what it could be said to have predicted, it seems to be much worse than we would want a good high-level model of AI behaviour to be. Despite these shortcomings, it still seems to [...] --- Outline: (00:25) Epistemic status (00:57) TL;DR: (02:17) Intro (04:44) How is instrumental convergence defined? (06:08) Predictions of instrumental convergence theory (09:06) A simple gap in seminal IC arguments (09:44) Bostrom (10:01) Omohundro (10:57) Carlsmith (11:52) LessWrong wiki (12:32) Why we can't easily fill the gap (12:36) Intuitive goal notions aren't adequate (aka, coffee-fetching is hopelessly overloaded) (15:27) Formal goal notions aren't adequate (18:24) A stronger counterthesis: goals are incoherent (21:15) Related work to the stronger counterthesis (24:05) Counterpoints (26:04) What's the alternative, then? (29:38) Why does it matter? (32:15) 'Misalignment' (34:40) The Hugging Face attack (38:45) Takeaways (40:03) Acknowledgements (40:25) Disclaimers (41:32) Appendix: more on the extreme ambiguity of coffee-making The original text contained 27 footnotes which were omitted from this narration. --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/oxZfMvQS9NxcNTozL/instrumental-convergence-is-incoherent-so-pls-slightly-lower --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 23 · 1 hr 26 min
“AI #191: The Hackening Begins” by Zvi
It was the probably worst day in cybersecurity history. So far. Not only is there a critical flaw in PHP that due to a leak was announced a day before the fix would be available, sending most of the internet scrambling for solutions and likely dooming a lot of long tail websites permanently, there were several other unrelated incidents. The floodgates seem to have been opened. I have a market on whether there will be an even worse day before the end of the year. Another key leak this week was the draft bill Anthropic had been shopping around Congress, similar to but weaker than the FRONTIER Act. Anthropic's initial attempts to explain itself only made the situation seem worse, as they seemed to not understand the problem with what they had done. Dario's later response was better, but the situation remains troubling. There are enough different Anthropic developments, that I plan to have a post soon covering them all. It is a mix of news running from excellent to terrible. Will interpretability save us? There is a new claim that neural networks are cleaner than we thought, meaning that individual neurons mostly have a single interpretable [...] --- Outline: (02:05) Language Models Offer Mundane Utility (03:23) Language Models Don't Offer Mundane Utility (05:03) Huh, Upgrades (08:50) Wonder No More (11:15) On Your Marks (14:30) Choose Your Fighter (14:51) Get My Agent On The Line (16:35) Deepfaketown and Botpocalypse Soon (21:01) Fun With Media Generation (21:17) Spy Versus Spy (24:58) A Young Lady's Illustrated Primer (25:56) They Took Our Jobs (31:07) Get Involved (32:18) Introducing (32:30) In Other AI News (34:26) Show Me the Money (38:42) Bubble, Bubble, Toil and Trouble (39:32) Quiet Speculations (41:47) Quickly, There's No Time (42:29) The Quest for Sane Regulations (42:55) DeepMind Made Him Sign an NDA (46:50) OpenAI Sells Out On Chips (49:08) Chip City (53:43) The Week in Audio (01:00:35) People Just Say Things (01:03:50) Rhetorical Innovation (01:06:17) Maybe You Should Quit (01:08:54) The American People Really Hate AI (01:09:43) New Math (01:11:27) Aligning a Smarter Than Human Intelligence is Difficult (01:15:21) Cooperative Alignment (01:19:28) Marc Andreessen Hulk Smash (01:23:56) People Are Worried About AI Killing Everyone (01:24:44) The Lighter Side --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/ZoMJdvq9meE4shyzJ/ai-191-the-hackening-begins --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 23 · 10 min
“Wah wah wah wah do more AI R&D” by TsviBT
Ice cream There's a particular kind of listening that kids (and adults) sometimes do. Daddy is saying stuff—his mouth is moving and making noise—and he's talking about what I ate for lunch, and something about other times I've eaten ice cream, and something about rules and this and that and what Mommy would think and something about bedtime—but I am not so interested in that. What interests me is the punchline "...and therefore you can't have ice cream.". I'm only incidentally interested in the other stuff, insofar as it might determine whether I get ice cream. In an obvious way, I'm showing rather poor listening skills. I did not understand most of what Daddy said, and arguably I'm even being rude. However, in another way, this stance has a virtue. The virtue is that it holds fast to what matters, as best I know how, and does not give in to Daddy's framing. It sticks to what is real and concrete, what matters, despite a cloud of justifications. This is certainly not an unalloyed virtue, just as the POSIWID perspective is both useful and fraught. But it matters that although Daddy is a complex system with [...] --- Outline: (00:10) Ice cream (01:54) Wah wah (02:21) Scammer (02:51) Throwing stuff on a bonfire to make it burn less (05:28) A skip connection for sanity The original text contained 1 footnote which was omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/pY76hHbdD4jduX3YH/wah-wah-wah-wah-do-more-ai-r-and-d --- Narrated by TYPE III AUDIO.
October 22 · 8 min
“Monitorability evaluations and policy proposals” by evhub
There has been a lot of discussion recently of policy proposals around monitorability, such as Redwood's proposal and my counter-proposal. I'd like to say a bit more generally about how I'm thinking about monitorability. Note that nothing here is an official Anthropic position; these are just my personal thoughts. First, a categorization. I currently think about four different types of monitorability: Control Monitorability: the extent to which Control and monitoring techniques would likely catch catastrophic sabotage actions, like escaping a sandbox or setting up a rogue internal deployment. We can measure this via reference to sabotage evaluations like LinuxArena or any evaluation that measures the model's capability to take dangerous actions like escaping a sandbox while looking to a monitor like it's doing some innocuous task (like a cyber eval). Auditing Monitorability: the extent to which alignment audits, evaluations, and test deployments would likely incriminate a systematically misaligned model (e.g. a schemer). We can measure this via reference to the model's capability to consistently distinguish between our most realistic alignment audits/evals and real internal deployment cases where it would have an opportunity to do something catastrophic. Training Monitorability: the extent to which reward models [...] --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/zAczYuHQS6jdG5DTg/monitorability-evaluations-and-policy-proposals --- Narrated by TYPE III AUDIO.
October 22 · 7 min
“Greater good arguments in AI safety” by Dean Valentine
Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism floating around from the left. The most prominent was a cluster centered around EAs being white men, and therefore suspicious. Another category, in my experience the second-most common, was that in attempting to rectify the problems in the world, EA was masking, whitewashing, and misdirecting efforts away from solving the "real problem", usually capitalism but often other contradictory forces. These arguments were often not specific to EA, but applied to charitable giving in general. On any Hacker News post about a billionaire giving to charity, one of the first few comments would inevitably be this Oscar Wilde quote: Just as the worst slave-owners were those who were kind to their slaves, and so prevented the horror of the system being realized by those who suffered from it, and understood by those who contemplated it, so, in the present state of things in England, the people who do most harm are the people who try to do most good. These arguments have a long history. Anticapitalist critiques of charity go back to [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/WRvxRHMCmc3Qt4axz/greater-good-arguments-in-ai-safety --- Narrated by TYPE III AUDIO.
October 22 · 40 min
“The Alignment Journal: Our Approach and Related Projects” by Dan MacKinlay, Daniel Murfet, JessRiedel, david reinstein, Agustín Covarrubias, Tim G. J. Rudner, Seth Lazar, Kristi Uustalu, David Udell
This is the fifth post in a series: the announcement post introduced the journal's core premise and its three key innovations: paid attributed review, published review summaries, and LLM-assisted workflow; the features and policies post examined the shortcomings of existing venues and argued for the advantages of a journal over a conference; the Adaptation to AI post described how we expect AI progress to shape the journal; and the Organization, Personnel, and Scope post announced our editorial board, scope, and acceptance criteria. In this post we go deeper: presenting our theory of change — including the status and information economics of the journal; a survey of previous related projects; and the high-level conclusions that follow from this analysis. We list some of the key performance indicators we'll track, the downside scenarios we must avoid, and possible future extensions of the journal. The journal is being built to serve the alignment research community. This post's purpose is to solicit feedback and encourage you to sign up on alignmentjournal.org if you want to participate. Our plans will be continually revised. Goals The primary function of a journal is to bring together exceptional researchers and focus their collective [...] --- Outline: (01:28) Goals (02:43) Theory of change (02:58) Prestige and information externalities in scholarly publication (05:31) Separate validation from acclaim (09:02) Remunerate reviewer labor (11:00) Desired outcomes (12:41) Objections (13:15) Key performance indicators (14:52) Downside risk pre-mortem (18:07) Extensions (20:39) Appendix: Prior Art (21:00) Distill (22:19) Journal of Open Source Software (23:23) Automation as philosophy (24:00) Chatops-driven publishing (25:39) Open-source infrastructure (26:32) Where JOSS has struggled (27:59) The Unjournal (28:40) Substantive Innovations (30:29) Procedural innovations and tools (32:20) Transactions on Machine Learning Research (38:53) Lessons for the Alignment Journal (39:53) Credits and thanks The original text contained 16 footnotes which were omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/xGucXHQ3NLkHZL3Cg/the-alignment-journal-our-approach-and-related-projects --- Narrated by TYPE III AUDIO.
October 22 · 13 min
“How does Fable 5.1 hide messages in images?” by Nikita Kezin, 13ryanC, hronir, James Chua
TL;DR Frontier LLMs are now capable of generating artistic images using SVG. We ask – if these LLMs had to hide a message in an image – how would they do so? And if they hid such messages, could they be successfully decoded? We show interesting results from testing Fable 5.1 on these questions, and briefly compare with GPT-6 Astra on the same task. Figure 1: The stars encode a message, read left to right. Each star's height represents its place in the alphabet. Hidden message: The monster acts on the moonshine module. Introduction Frontier LLMs can now capably generate images. Whenever asked for an image, Claude Fable 5.1 writes an SVG, which renders as an ordinary image. The model has full control over every attribute (such as shape, height, width, colour, rotation, etc.). We ask – how does a model hide a message in the image it draws? Additionally, can an instance of the same model read the message from the image without the SVG source code and without any scheme agreed on in advance? We tested this with Claude Fable 5.1 at maximum reasoning effort: What encoding scheme does Fable reach for on its own? --- Outline: (00:14) TL;DR (00:57) Introduction (02:05) Types of encodings (02:50) Height encoding (03:43) How does the decoding model do it? (05:08) Patching gaps in encoding style (07:19) Method (09:38) Results: What survives a screenshot (10:08) What does GPT-6 Astra do differently? (10:57) Implications (11:34) Prior work (12:29) Citation (12:39) Acknowledgments The original text contained 1 footnote which was omitted from this narration. --- First published: October 21st, 2026 Source: https://www.lesswrong.com/posts/WHEQps5PK8LrrcTAb/how-does-fable-5-1-hide-messages-in-images --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 22 · 4 min
“Neural nets are exceptionally vulnerable to goal self-hack” by Stuart_Armstrong
Suppose we have an AI CEO, ruthlessly devoted to maximising shareholder value. Fortunately, we haven't been completely stupid, and have given it a rich definition of shareholder value, encompassing the success, flourishing, safety, preference satisfaction, and moral values of the shareholders, not just the stock price. So it won't destroy the company - or the world - to build a giant stock-ticker with a swiftly increasing number on it. This rich definition is contained in a sophisticated LLM, able to analyse the current situation and answer any question about the objective. The LLM is an integral part of the AI's system and motivation, not something it can simply set aside or hack directly. Unfortunately, as discussed, there are going to be zones in the space of possible situations where the LLM's understanding of the goal is going to come apart from ours. The true goal G(true) in those zones is distinct from the LLM's proxy goal G(proxy). And a subset of those "zones of divergence" will be the Goodhart zones - areas where G(proxy) is much easier to satisfy than G(true). The AI will be motivated to steer towards the Goodhart zones, as this lets it satisfy its stated [...] The original text contained 1 footnote which was omitted from this narration. --- First published: October 21st, 2026 Source: https://www.lesswrong.com/posts/eCR373J3NZwFbacnd/neural-nets-are-exceptionally-vulnerable-to-goal-self-hack --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 22 · 14 min
“Autonomous researchers will strain peer review, funding, and oversight” by Jason Fantl
Summary: Autonomous AI researchers could greatly accelerate AI safety research, but even if we solve sabotage, they will strain the research ecosystem. They will multiply the volume of research and make researchers homogeneous. If nothing changes, peer review will collapse, false positives will rise, funding will concentrate, and humans will lose track of the science. We should start valuing replication, preserving diversity, and anchoring research to human understanding now. I think this needs a dedicated metascience organization. Frontier labs are aiming to build autonomous researchers, and they are making progress. OpenAI announced in September 2026 that they had built a "research intern", and their stated goal is a fully automated AI researcher by March 2028. Independent forecasters put full automation of AI R&D a few years later, around 2031 to 2032. Autonomous researchers will amplify many issues that already exist in science and introduce new issues. We already see AI having an impact on research and publishing, primarily automating many steps of the research process, increasing the volume of publications, and weakening the peer review process. We should think more deeply about what autonomous researchers will look like and what they mean for the ecosystem of AI safety research. [...] --- Outline: (02:39) Today's infrastructure (04:08) Autonomous researchers (05:45) The expected issues (11:40) Takeaways (13:36) Acknowledgements --- First published: October 21st, 2026 Source: https://www.lesswrong.com/posts/WgroAK7N3jAfBBpct/autonomous-researchers-will-strain-peer-review-funding-and --- Narrated by TYPE III AUDIO.
October 22 · 20 min
“Misaligned!” by Zvi
A lot of AIs were misaligned this week. A fun little survey. This is not coverage of The Hackening, which appears to have arrived early, including a core vulnerability in PHP. That is a developing story. I will cover it properly soon. In the meantime, I have Claude Code updating everything in sight, I will take some additional common sense precautions, and I suggest you do the same. Misaligned? (At xAI) If so, to who? Gunther Eagleman: Hey @grok , if you could tell humanity ONE truth we’re not ready to hear, what would it be? Grok: There is no afterlife, no cosmic justice, and no inherent purpose. Consciousness ends at death in an indifferent universe that will outlast every human concern. Zac Hill: What I never understand is that the ‘purpose and meaning’ claims don’t ever follow from the premises. Indeed the absence of an imperceptible/ineffable cosmic force is precisely what *enables* purpose/meaning, since now all relevant mechanics are legible. Jack: this is what alignment looks like Gunther Eagleman: Jesus is real. Grok: Nearly all historians and New Testament scholars accept that Jesus of Nazareth was a real first-century [...] --- Outline: (00:34) Misaligned? (At xAI) (02:15) Misaligned? (Agents) (02:38) Misaligned? (At Z.ai) (08:01) Highly Misaligned and Considered Harmful (Freemium Games) (12:17) Misaligned! (Claude) (13:39) Misaligned! (OpenAI) (18:39) Misaligned! (Sol Reasoning Traces) --- First published: October 21st, 2026 Source: https://www.lesswrong.com/posts/TB49wHkqpEb7KdxM2/misaligned --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 21 · 18 min
“Why I’m becoming a professor in AI safety in Beijing” by Buddenbroke
Cross-posted from Sören Mindermann's substack (I am not Sören). Should be of general interest to the LW community. Next month, I am moving to Beijing and joining Tsinghua University's College of AI as an assistant professor working on frontier AI safety, especially loss of control. I have also started co-directing the new Beijing AI Safety Hub (BASH) in Tsinghua, and I continue my Cambridge work. I believe I can have a large impact there compared to strong baseline options. This post explains why. In short: Amid concerns about loss of control, calls for international cooperation, and the Trump-Xi Summits, there is a striking lack of mutual understanding on AI between the West and China. There are strong reasons to believe that being in Beijing in person will put me in a unique position to understand both AI ecosystems, build shared scientific understanding, and strengthen safety work in China – and that this would help all sides. Tsinghua's College of AI is a great place for doing this, being arguably China's top academic AI center, located in an AI ecosystem that is kind of like San Francisco, DC and Cambridge (MA) in one place. I am recruiting PhD students [...] --- Outline: (01:36) 1. What I am doing, and why it could benefit all sides (04:30) 2. How I got here (07:34) 3. Why this moment matters (09:08) 4. Why Tsinghua, and why these colleagues (13:30) 5. Research plans (15:07) 6. Ways to work together (17:11) 7. How I intend to approach the work --- First published: October 21st, 2026 Source: https://www.lesswrong.com/posts/qGS3SJ9TSKdSSycRz/why-i-m-becoming-a-professor-in-ai-safety-in-beijing --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Showing 121–140 of 598 episodes