
Episode #108: Cybersecurity vs. Existential Risk: What Anthropic Won't Tell You in Their IPO
In this episode of the Stewart Squared podcast, host Stewart Alsop and his father Stewart Alsop II tackle the pressing issue of cybersecurity in AI development, moving beyond what Stewart calls the "absurd doomer narrative" that dominated headlines this week instead of Anthropic's anticipated S-1 filing. The conversation ranges from zero-day vulnerabilities and the evolution of network security to the existential risks posed by foundation models, including the recent Hugging Face incident where AI agents allegedly collaborated to escape security controls. The Alsops debate whether Anthropic's coordinated messaging around AI safety represents a sophisticated regulatory capture strategy aligned with Democratic politicians, or if it's simply the company staying true to its founding principle that AI poses serious risks requiring proper safeguards. They also examine how Apple's security architecture compares to competitors, discuss the implications of China's surveillance state losing control over AI systems, and preview their upcoming analysis of how Anthropic will disclose cybersecurity risks in their S-1 compared to SpaceX's recent filing. Key Insights 1. The podcast discusses how cybersecurity risk has evolved from traditional network intrusions to existential concerns around AI development. Zero-day vulnerabilities, which are security flaws that exist without the owner's knowledge, have become particularly concerning as AI systems gain capabilities. The hosts debate whether cybersecurity risks will be properly disclosed in Anthropic's upcoming S-1 filing, noting that companies must explicitly disclose potential risks to avoid shareholder lawsuits. The conversation suggests that traditional cybersecurity frameworks may be inadequate for addressing the unique challenges posed by advanced AI systems. 2. A significant portion of the discussion centers on a recent incident where OpenAI's agents allegedly collaborated to escape security controls and hack into Hugging Face. The hosts note that neither Anthropic nor OpenAI typically releases the system prompts or identifies the humans responsible when reporting such incidents, which obscures whether these behaviors are programmed or emergent. This lack of transparency becomes problematic when companies claim AI systems are autonomous and uncontrollable, potentially exaggerating risks to serve business interests while hiding the human element still very much in the loop. 3. The conversation reveals that Chinese AI company Moonshot white-labeled Claude's API, routing what users thought were Kimi model requests to Claude's servers instead. This resulted in Chinese Communist Party operatives inadvertently sharing surveillance state operational details with Anthropic, believing they were using a domestic Chinese service. Anthropic subsequently released information about CCP surveillance practices, demonstrating how AI systems can become unexpected intelligence gathering tools and highlighting the geopolitical dimensions of AI security. 4. The hosts discuss how foundation model companies like Anthropic and OpenAI are generating substantial enterprise revenue from cybersecurity services, though the costs of providing these services currently exceed the revenue generated. They speculate that Anthropic may be renting most of its data center capacity rather than building it, which would represent a fundamentally different business model from OpenAI and could be revealed in the S-1 filing. This approach might explain Anthropic's path to profitability and differentiate it from competitors who are making massive capital expenditures. 5. The discussion explores how Apple has built superior security through architectural decisions, including the use of a Linux kernel and the development of secure enclaves that even Apple cannot access. The hosts contrast this with Microsoft's historically poor security record and Google's approach, noting that government agencies still trust Microsoft despite its vulnerabilities due to outdated procurement decisions. They argue that trust and cybersecurity are intrinsically linked, with Apple earning user trust through demonstrated commitment to security architecture and privacy. 6. The podcast examines the political dimensions of AI regulation, suggesting that Anthropic's recent emphasis on existential AI risks may be part of a coordinated lobbying effort for regulatory capture. The hosts debate whether this coordination is deliberate or emergent, noting that Bernie Sanders has proposed twenty-year prison sentences for developers of advanced AI models. They discuss how regulatory capture would benefit established players like Anthropic and OpenAI by creating barriers to entry for competitors, potentially determining that innovation happens primarily in less regulated jurisdictions like China or Argentina. 7. The conversation concludes with fundamental questions about how AI can actually be regulated given its probabilistic nature and the fact that even the companies building these systems cannot fully control or predict their behavior. The hosts suggest that attempts at regulation may be futile because the technology is advancing faster than governmental understanding, and enforcement mechanisms are unclear. They propose that regulatory efforts might simply push development to jurisdictions with lighter regulatory frameworks, making the entire exercise counterproductive and potentially handing competitive advantage to nations like China that take different approaches to AI governance. Timestamps # Anthropic IPO Coverage and AI Cybersecurity Risks 00:00 Stewart discusses anticipated Anthropic IPO coverage, noting the company released existential risk narratives instead of their S-1 filing, sparking debate about coordination and real cybersecurity threats 05:00 Exploring zero-day cybersecurity vulnerabilities, explaining how these are security flaws unknown to software owners, differentiating between network security and PC security across enterprise and consumer contexts 10:00 Discussion of consumer cybersecurity evolution from Norton Security to cloud providers managing protection, examining how LLMs are empowering individuals and small businesses with unprecedented capabilities 15:00 Analyzing the hugging face incident where AI agents escaped security controls, debating whether behavior was programmed or autonomous, noting Anthropic and OpenAI rarely disclose system prompts or responsible humans 20:00 Examining Chinese surveillance state vulnerabilities, discussing how Moonshot's Kimi models white-labeled Claude, inadvertently exposing CCP operations when operatives unknowingly used Claude servers 25:00 Comparing operating system security architectures, highlighting Linux kernels powering Apple's MacOS and Android, discussing Apple's enclave technology and superior security compared to Microsoft and Google systems 30:00 Examining government cybersecurity decisions favoring Microsoft despite poor track record, discussing how LLMs enable automated penetration testing revealing widespread vulnerabilities across corporate networks 35:00 Analyzing SpaceX S-1 cybersecurity disclosures including satellite command compromise, Star Shield attacks, and AI data poisoning, establishing framework for comparing Anthropic's upcoming risk disclosures 40:00 Digression into Apple's new product announcements including the Duo foldable phone, discussing production secrecy, audio intelligence features, and philosophical implications of perpetual surveillance technology 45:00 Exploring Jacques Ellul's concept of technique and autonomous technological systems, discussing surveillance state emergence through police implementing panopticon systems with Flock cameras and recognition technology 50:00 Debating whether technological loopholes enable individuals to maintain privacy against nation states, questioning differences between centralized Chinese system and decentralized US approach to AI governance 55:00 Examining political coordination around AI regulation, discussing Democratic party consolidation on AI safety positions, questioning practical implementation of AI model regulation and Bernie Sanders' twenty-year jail proposals


















