Google Built Two New TPUs Without New Silicon: The 8i Max and 8TX
Austin Lyons and Vik Sekar break down Google's two new TPU systems, the 8i Max and 8TX. These are not new silicon but new system configurations of the existing V8 chip, designed for specific workloads. The conversation covers the 8i Max's use of advanced packaging to increase density and the 8TX's swap to an AMD CPU to enable confidential computing for multi-tenant cloud customers. Presented by Crusoe: Fine-tune and serve open models without managing a single GPU. Deploy on Crusoe in one click, or download and run it anywhere. Upload a dataset, pick an open model, and pay per token rather than per GPU hour. Accounts are free, and get $5 in credit when you sign up. Try Serverless Fine-Tuning: https://semidoped.com/crusoe Key Takeaways: - Google's new TPUs are system-level innovations, not new silicon. The 8i Max and 8TX are both built around the existing V8 chip, signaling a strategy of diversifying offerings around a single generation. - The 8i Max is a packaging play. It places two 8i chips on one module to double on-chip SRAM to ~768 MB, creating a single logical processor for long-context and agentic workloads. - The 8TX system is built for security, not performance. It combines 8T training silicon with an inference network (Boardfly) and an AMD EPYC CPU to deliver confidential computing via AMD's SEV technology. - The choice of AMD's EPYC CPU over Google's own Axion is a time-to-market decision. It gives Google's cloud customers a mature, flight-tested hardware security feature without waiting for an in-house version. - The 'Max' branding signals a roadmap. Austin and Vik speculate that this year's premium dual-die 8i Max configuration will likely become the baseline for next year's TPU 9i. - Software unlocks the hardware's value. The new TPU Sync layer manages the memory hierarchy (HBM, DRAM, SSD), cutting KV cache transfer overhead by 90% and improving time-to-first-token. This episode is supported by Meter. You have the GPUs and the power. Now who builds the network, and who do you call when it breaks? Meter designs, deploys, and runs frontend, storage, and management networks on its own switches and its own network OS. One call reaches the team that designed the hardware and wrote the software. https://semidoped.com/meterGoogle's two new TPU systems, the 8i Max and 8TX are not new silicon but new system configurations of the existing V8 chip. The 8i Max uses advanced packaging to increase density and the 8TX swaps to an AMD CPU to enable confidential computing for multi-tenant cloud customers. Presented by Crusoe: Fine-tune and serve open models without managing a single GPU. Deploy on Crusoe in one click, or download and run it anywhere. Upload a dataset, pick an open model, and pay per token rather than per GPU hour. Accounts are free, and get $5 in credit when you sign up. Try Serverless Fine-Tuning: https://semidoped.com/crusoe Key Takeaways: - Google's new TPUs are system-level innovations, not new silicon. The 8i Max and 8TX are both built around the existing V8 chip, signaling a strategy of diversifying offerings around a single generation. - The 8i Max is a packaging play. It places two 8i chips on one module to double on-chip SRAM to ~768 MB, creating a single logical processor for long-context and agentic workloads. - The 8TX system is built for security, not performance. It combines 8T training silicon with an inference network (Boardfly) and an AMD EPYC CPU to deliver confidential computing via AMD's SEV technology. - The choice of AMD's EPYC CPU over Google's own Axion is a time-to-market decision. It gives Google's cloud customers a mature, flight-tested hardware security feature without waiting for an in-house version. - The 'Max' branding signals a roadmap. Austin and Vik speculate that this year's premium dual-die 8i Max configuration could become the baseline for next year's TPU 9i. - Software unlocks the hardware's value. The new TPU Sync layer manages the memory hierarchy (HBM, DRAM, SSD), cutting KV cache transfer overhead by 90% and improving time-to-first-token. This episode is supported by Meter. You have the GPUs and the power. Now who builds the network, and who do you call when it breaks? Meter designs, deploys, and runs frontend, storage, and management networks on its own switches and its own network OS. One call reaches the team that designed the hardware and wrote the software. https://semidoped.com/meter Chapters: 0:00 Not New Silicon, New Systems 7:28 Refresher: 8T vs. 8i 12:42 Defining the 8i Max 13:59 The 'Mirror Chip' Mistake 18:47 Packaging vs. Optics 22:42 The 'iPhone' Naming Strategy 23:09 Defining the 8TX 28:51 How Confidential Computing Works 30:03 Why AMD? SEV Technology 39:58 Fine-Tuning on the 8TX 42:37 TPU Sync: The Software Layer 51:48 Dogfooding as an Advantage Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free: https://daily.semidoped.com/ Chapters: 0:00 Not New Silicon, New Systems 7:28 Refresher: 8T vs. 8i 12:42 Defining the 8i Max 13:59 The 'Mirror Chip' Mistake 18:47 Packaging vs. Optics 22:42 The 'iPhone' Naming Strategy 23:09 Defining the 8TX 28:51 How Confidential Computing Works 30:03 Why AMD? SEV Technology 39:58 Fine-Tuning on the 8TX 42:37 TPU Sync: The Software Layer 51:48 Dogfooding as an Advantage Follow Chipstrat: Newsletter: https://www.chipstrat.com X: https://x.com/chipstrat Follow Vik: Newsletter: https://www.viksnewsletter.com/ X: https://x.com/vikramskr Follow Semi Doped: Get more of Austin and Vik daily, free: https://daily.semidoped.com/
- Transcript

