Protocol Learning, Decentralized Frontier Risk and the No-Off Problem
Not on the map. Why?
Abstract (arXiv)
Frontier models are currently developed and distributed primarily through two channels: centralized proprietary APIs or open-sourcing of pre-trained weights. We identify a third paradigm - Protocol Learning - where models are trained across decentralized networks of incentivized participants. This approach has the potential to aggregate orders of magnitude more computational resources than any single centralized entity, enabling unprecedented model scales and capabilities. However, it also introduces novel challenges: heterogeneous and unreliable nodes, malicious participants, the need for unextractable models to preserve incentives, and complex governance dynamics. To date, no systematic analysis has been conducted to assess the feasibility of Protocol Learning or the associated risks, particularly the 'No-Off Problem' arising from the inability to unilaterally halt a collectively trained model. We survey recent technical advances that suggest decentralized training may be feasible - covering emerging communication-efficient strategies and fault-tolerant methods - while highlighting critical open problems that remain. Contrary to the notion that decentralization inherently amplifies frontier risks, we argue that Protocol Learning's transparency, distributed governance, and democratized access ultimately reduce these risks compared to today's centralized regimes.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 21 of its 39 references that are in the map. First posted on arXiv 10 Dec 2024.
Builds on
21 of 39 references.
References · 21
- 2017Attention Is All You Needcited by 60,361
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
- 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
- 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
- 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
- 2023AI Alignment: A Comprehensive Surveycited by 263
- 2023An Overview of Catastrophic AI Riskscited by 222
9 more
- 2017Asynchronous Decentralized Parallel Stochastic Gradient Descentcited by 159
- 2023Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigationscited by 158
- 2023SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficientcited by 59
- 2021Distributed Deep Learning in Open Collaborationscited by 58
- 2020Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Expertscited by 43
- 2021Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devicescited by 36
- 2016Gossip training for deep learningcited by 35
- 2020Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selectioncited by 20
- 2019A scalable verification solution for blockchainscited by 10
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2025Distributed and Decentralised Training: Technical Governance Challenges in a Shifting AI Landscapecited by 3
- 2025You Need Reasoning to Learn Reasoning: The Limitations of Label-Free RL in Weak Base Modelscited by 2
- 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocolcited by 0
- 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocol23 May 2026
- 2025You Need Reasoning to Learn Reasoning: The Limitations of Label-Free RL in Weak Base Models7 Nov 2025
- 2025Distributed and Decentralised Training: Technical Governance Challenges in a Shifting AI Landscape10 Jul 2025