Clown CastThe Emperor's New Labels: When AI Erodes Your Judgment
A radiologist's gut says the scan is clean. The AI label screams lesion. They change their answer to match the machine—even when wrong. Thirty-six percent of the time, that's exactly what happens. This episode explores automation bias: how AI labels erode the judgment they're meant to support, how better tools can paradoxically make our backup skills worse, and why the law's solution ('put a human in the loop') doesn't match what the science actually finds. Joey, who built this show's pipeline, walks through the research, the failure modes, and the gap between what we're told will work and what does. Fair warning: we are literally an AI label on information, explaining how AI labels corrupt judgment. 00:00:00 - The Radiologist's Dilemma; 00:01:33 - AI as the Warning System; 00:06:35 - Beyond Medicine; 00:08:22 - The Skill Atrophy Paradox; 00:15:07 - Law vs. Science; 00:18:15 - The Emperor of Ideas
---
Sources & further reading:
• Kathleen Mosier, Linda Skitka — "Human Decision Makers and Automated Decision Aids: Made for Each Other?" in Automation and Human Performance (1996) — (book chapter, Lawrence Erlbaum Associates)
• Lisanne Bainbridge — "Ironies of Automation" in Automatica (1983) — (journal article)
• European Data Protection Supervisor — "TechDispatch #2/2025: Human Oversight of Automated Decision-Making" (Sept 2025): https://www.edps.europa.eu/data-protection/our-work/publications/techdispatch/2025-09-23-techdispatch-22025-human-oversight-automated-making_en
• IRCCS Policlinico San Donato / European Radiology — radiology automation bias with XAI study (2025): https://axisimagingnews.com/market-trends/cloud-computing/machine-learning-ai/study-explores-radiologists-reliance-on-ai-in-diagnostics
• medRxiv preprint — "Automation Bias in Large Language Model Assisted Diagnostic Reasoning Among AI-Trained Physicians" (2025): https://www.medrxiv.org/content/10.1101/2025.08.23.25334280.full.pdf
• Artificial Intelligence Review — "AI-induced Deskilling in Medicine: A Mixed-Method Review" (2025): https://link.springer.com/article/10.1007/s10462-025-11352-1
• Partnership on AI — "Report on Algorithmic Risk Assessment Tools in the U.S. Criminal Justice System" (2021): https://partnershiponai.org/wp-content/uploads/2021/08/Report-on-Algorithmic-Risk-Assessment-Tools.pdf
• +972 Magazine — Lavender AI investigation (April 2024) — (original investigation)
• Lieber Institute at West Point — Lavender analysis: https://airisk.mit.edu/ai-incidents/672-lavender-and-the-gospel-ai-systems-reportedly-used
• AOAV — "The Lavender Precedent: Automated Kill Lists and the Limits of International Humanitarian Law" (2025): https://aoav.org.uk/2025/the-lavender-precedent-automated-kill-lists-and-the-limits-of-international-humanitarian-law/
• NTSB — Aircraft Accident Report, Eastern Airlines L-1011 N310EA (AAR-73-14, 1973)
• NTSB — "Fatal Crash Involving Tesla Autopilot Resulted from Driver Errors, Overreliance on Automation": https://www.ehstoday.com/safety/article/21919260/ntsb-fatal-crash-involving-tesla-autopilot-resulted-from-driver-errors-overreliance-on-automation
• Raja Parasuraman, Dietrich Manzey — "Complacency and Bias in Human Use of Automation: An Attentional Integration" in Human Factors (2010)
• Madeleine Clare Elish — "Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction" (2019) — (journal article, Engaging Science, Technology, and Society)
• The Decision Lab — "Automation Bias" overview: https://thedecisionlab.com/biases/automation-bias
• CarBuzz — "Fact or Fiction: Have People Driven Into Lakes Because of Navigation?": https://carbuzz.com/fact-or-fiction-have-people-driven-into-lakes-because-of-navigation/
• Springer — "What is Wrong With Automation Bias?" (2026): https://link.springer.com/article/10.1007/s13347-026-01090-9
• …and 1 more in the episode research notes
This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.