Quite a long technical paper, but this kind of research, I think, is worth highlighting. I understand that this is not most people’s cup of tea, but I’m putting it out into audio for those that think it is a value add.
If you think this kind of accessibility boosting is important, I ask for a couple of things. One, share with your friends. Two, give me ideas for other such papers/reports that might be worthwhile converting into full-cast produced audio.
I’m also very much open to working with people and / or organisations to get audio like this made for their articles and papers. I’m willing and happy to consult on audio production pipeline stuff or anything of that nature in producing episodes like these Please feel free to reach out to me, either here or via hello@askwhocasts.com.
If world leaders decided tomorrow to pause frontier AI training, could they actually do it? This book-length working paper from an interdisciplinary group of Berkeley-led researchers, including Stuart Russell and Rose Gottemoeller, takes the question seriously and answers it in detail: a “hardwired pause” that freezes a whitelist of today’s models into inference-only chips, takes a global census of the training-capable hardware already out there, governs the concentrated chip supply chain, and asks how much compute a cheater could hide, how long the pause could last, what it would cost the economy, and what would have to change in Washington and Beijing for leaders to want it. Citations and cross-references have been left out for listening; the full paper, with its references and calculations, is at the link.
* 00:00 - Introduction
* 02:06 - 1 Introduction
* 07:40 - 1.1 A Hardwired Pause
* 17:59 - 1.2 Scope
* 22:06 - 1.3 Organization
* 23:11 - 2 Background: arguments for a pause and lessons for AI governance
* 23:34 - 2.1 Observed arguments for a pause
* 24:02 - 2.1.1 Loss of control
* 26:10 - 2.1.2 Malicious use and catastrophic harms
* 27:32 - 2.1.3 Economic disruption, gradual disempowerment, and concentration of power
* 29:27 - 2.1.4 Destruction of a healthy information ecosystem
* 30:41 - 2.1.5 Geostrategic instability
* 32:08 - 2.1.6 Why a pause
* 33:21 - 2.2 Historical lessons for AI governance
* 34:32 - 2.2.1 Lesson 1: Governance is a three-legged stool
* 36:26 - 2.2.2 Lesson 2: Governance agreements tend to focus on observable, verifiable units
* 38:58 - 2.2.3 Lesson 3: Monitoring needn’t be perfect to be effective
* 40:07 - 2.2.4 Lesson 4: Signatories need to benefit
* 43:09 - 2.2.5 Lesson 5: Effective Institutions Are Needed
* 49:28 - 2.3 Game-theoretic lessons for AI governance
* 55:22 - 3 Example implementation of a hardwired pause
* 55:27 - 3.1 Overview and objectives
* 01:02:56 - 3.1.1 Political objectives
* 01:07:40 - 3.1.2 Technical objectives
* 01:11:56 - 3.1.3 Technological readiness
* 01:12:47 - 3.2 Operationalizing technical requirements
* 01:13:33 - 3.2.1 Inference-only and training-capable accelerators
* 01:20:41 - 3.2.2 Quantitative training thresholds
* 01:30:13 - 3.2.3 Verification and breakout accounting
* 01:35:50 - 3.2.4 Model whitelisting
* 01:42:20 - 3.3 Hardware governance
* 01:42:58 - 3.3.1 Residual uses of training-capable accelerators
* 01:46:37 - 3.3.2 Hardware licensing, registration, and tracking
* 01:54:11 - 3.3.3 Sociotechnical verification measures for preventing illicit use of residual training-capable accelerators
* 02:01:03 - 3.3.4 Governing hardware production
* 02:05:01 - 3.4 Transition governance
* 02:05:17 - 3.4.1 Economic alignment
* 02:07:50 - 3.4.2 Phase 1 (design)
* 02:11:55 - 3.4.3 Phase 2 (substitution)
* 02:13:27 - 3.4.4 Traded-in training-capable compute
* 02:15:00 - 3.5 Possible exit conditions
* 02:17:53 - 3.6 Institutional governance
* 02:27:53 - 4 Feasibility of inference-only hardware
* 02:28:14 - 4.1 Quantifying training capability
* 02:32:05 - 4.2 Practical inference-only accelerators
* 02:36:57 - 4.3 Model-specific integrated circuits (MSICs)
* 02:45:00 - 4.4 Time required to ramp inference-only hardware production
* 02:54:32 - 5 Feasibility of hardware governance
* 02:54:36 - 5.1 Overview
* 02:58:17 - 5.2 The supply chain for accelerator chips
* 03:06:35 - 5.3 Feasibility of governing chip manufacturing
* 03:07:14 - 5.3.1 Feasibility of detecting undeclared facilities
* 03:10:28 - 5.3.2 Feasibility of monitoring declared manufacturing facilities
* 03:12:21 - 5.4 Feasibility of a global accelerator census
* 03:14:23 - 5.4.1 The paper trails left by pre-pause hardware
* 03:23:02 - 5.4.2 Incentives of states and private actors
* 03:28:01 - 5.5 Estimating how many chips a census would miss
* 03:33:51 - 5.5.1 Estimating the quantity of pre-pause hardware
* 03:40:47 - 5.5.2 Estimating the missing fraction by holding type
* 03:41:22 - Missing failed chips
* 03:44:00 - Functional chips not declared
* 03:48:23 - 6 Feasibility of verification
* 03:48:27 - 6.1 Summary assessment
* 03:55:04 - 6.2 Fleet thresholds for defection
* 04:01:31 - 6.2.1 Method for determining governance thresholds
* 04:11:38 - 6.2.2 Covert evasion fleet threshold
* 04:21:30 - 6.2.3 Overt breakout fleet threshold
* 04:29:28 - 6.2.4 Private evasion fleet threshold
* 04:35:15 - 6.2.5 Small-scale evasion
* 04:38:16 - 6.3 Training efficiency gains and governance innovation
* 04:38:47 - 6.3.1 Estimates of recent historical training efficiency gains
* 04:42:40 - 6.3.2 Training efficiency gains after a pause
* 04:46:34 - 6.4 Quantitative ledger estimates
* 04:47:42 - 6.4.1 Discount factors
* 04:53:33 - 6.4.2 Transitional training-capable inference service
* 04:58:52 - 6.4.3 Dark compute
* 05:06:44 - 6.4.4 Inference-only accelerators
* 05:10:44 - 6.4.5 Consumer GPUs
* 05:15:28 - 6.4.6 Licensed sovereign fleets
* 05:18:49 - 6.4.7 Other categories
* 05:21:03 - 6.5 Durability assessment
* 05:24:28 - 6.5.1 Durably guarding against covert evasion
* 05:26:48 - 6.5.2 Durably guarding against overt breakout
* 05:29:44 - 6.5.3 Extension options
* 05:30:29 - 6.5.4 Lithography proliferation
* 05:31:39 - 7 Implications of a hardwired pause
* 05:32:26 - 7.1 Would restraints on frontier AI crash the economy?
* 05:47:02 - 7.2 Reduction of inference costs and related economic effects
* 05:49:40 - 7.3 Economics of MSICs
* 05:52:21 - 7.3.1 MSIC adoption in a mixed MSIC-flexible world
* 06:06:37 - 7.3.2 The cost of a pure MSIC world
* 06:10:44 - 7.3.3 Shifting demand among whitelisted models
* 06:17:12 - 7.3.4 Temporary or permanent de-whitelisting of models
* 06:26:53 - 7.4 Impacts on concentration of power
* 06:31:46 - 7.5 Consequences of receding knowledge cutoffs
* 06:37:47 - 7.6 The need for other measures
* 06:41:37 - 8 Pause-willing futures: conditions favoring international AI restraint
* 06:42:02 - 8.1 Introduction
* 06:45:48 - 8.2 Theoretical drivers of pause willingness
* 06:47:15 - 8.2.1 Fundamental changes in the technology
* 06:54:03 - 8.2.2 Changes in the international system: cooperation in the shadow of great power rivalry
* 06:55:40 - Power transition theory and the security dilemma—heightened dangers open up possibilities
* 06:59:31 - Hegemonic stability and the strategic value of institutional governance
* 07:02:10 - Polarity and possibilities for collective action
* 07:05:52 - 8.2.3 Changes in national political environments
* 07:09:58 - 8.3 More than willingness: conditions for durability
* 07:14:50 - 8.4 Imagining pause willing futures
* 07:15:32 - 8.4.1 The expected costs of continued development have increased
* 07:18:53 - 8.4.2 The perceived value of “winning the AI race” has fallen
* 07:25:18 - 8.4.3 The distribution of capabilities creates support for a pause
* 07:26:20 - Limitations of the conventional wisdom
* 07:28:03 - When breaking new ground is hard and catching up is easy
* 07:31:09 - Trust dilemmas vs. Prisoner’s Dilemmas
* 07:32:24 - 8.4.4 Domestic coalitions could make reciprocal restraint politically viable
* 07:36:47 - 8.4.5 Shifting power distribution and heightened risks drive multilateral restraint
* 07:38:28 - 8.5 Major actors’ perspectives and incentives
* 07:40:37 - 8.5.1 The view from the United States
* 07:48:32 - 8.5.2 The view from the People’s Republic of China
* 07:56:15 - 9 Conclusion
* 07:58:47 - Reproducibility
* 07:59:22 - Financial Disclosures
* 08:00:49 - AI Use Disclosure
https://hardwired-pause.ai/paper
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit askwhocastsai.substack.com/subscribe