# What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring

arXiv: https://arxiv.org/abs/2303.11341 · HTML: https://mlsanity.com/paper/2303.11341/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 69 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU](https://mlsanity.com/paper/2303.06865/) (2023)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [Unsolved Problems in ML Safety](https://mlsanity.com/paper/2109.13916/) (2021)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [Predictability and Surprise in Large Generative Models](https://mlsanity.com/paper/2202.07785/) (2022)
- [The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation](https://mlsanity.com/paper/1802.07228/) (2018)
- [Compute Trends Across Three Eras of Machine Learning](https://mlsanity.com/paper/2202.05924/) (2022)
- [Decentralized Training of Foundation Models in Heterogeneous Environments](https://mlsanity.com/paper/2206.01288/) (2022)
- [Proof-of-Learning: Definitions and Practice](https://mlsanity.com/paper/2103.05633/) (2021)
- [Demystifying the Draft EU Artificial Intelligence Act](https://mlsanity.com/paper/2107.03721/) (2021)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient](https://mlsanity.com/paper/2301.11913/) (2023)
- [Scaling Laws for Deep Learning](https://mlsanity.com/paper/2108.07686/) (2021)
- ["Adversarial Examples" for Proof-of-Learning](https://mlsanity.com/paper/2108.09454/) (2021)
- [On the Replicability and Reproducibility of Deep Learning in Software Engineering](https://mlsanity.com/paper/2006.14244/) (2020)
- [Algorithmic progress in computer vision](https://mlsanity.com/paper/2212.05153/) (2022)
- [Regulating Artificial Intelligence: Proposal for a Global Solution](https://mlsanity.com/paper/2005.11072/) (2020)
- [Computational Power and the Social Impact of Artificial Intelligence](https://mlsanity.com/paper/1803.08971/) (2018)

## Built on by (most cited first, 21 of 21)

- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023)
- [AI capabilities can be significantly improved without expensive retraining](https://mlsanity.com/paper/2312.07413/) (2023)
- [Tools for Verifying Neural Models' Training Data](https://mlsanity.com/paper/2307.00682/) (2023)
- [Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models](https://mlsanity.com/paper/2412.17128/) (2024)
- [Verification methods for international AI agreements](https://mlsanity.com/paper/2408.16074/) (2024)
- [Trust AI Regulation? Discerning users are vital to build trust and effective AI regulation](https://mlsanity.com/paper/2403.09510/) (2024)
- [International Governance of Civilian AI: A Jurisdictional Certification Approach](https://mlsanity.com/paper/2308.15514/) (2023)
- [An International Consortium for Evaluations of Societal-Scale Risks from Advanced AI](https://mlsanity.com/paper/2310.14455/) (2023)
- [Towards Responsible Governance of Biological Design Tools](https://mlsanity.com/paper/2311.15936/) (2023)
- [A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Search](https://mlsanity.com/paper/2507.00004/) (2025)
- [Detecting Hidden ML Training With Zero-Overhead Telemetry](https://mlsanity.com/paper/2606.19262/) (2026)
- [Taking control: Policies to address extinction risks from advanced AI](https://mlsanity.com/paper/2310.20563/) (2023)
- [PoTS: Proof-of-Training-Steps for Backdoor Detection in Large Language Models](https://mlsanity.com/paper/2510.15106/) (2025)
- [What Is AI Safety? What Do We Want It to Be?](https://mlsanity.com/paper/2505.02313/) (2025)
- [Zero knowledge verification for frontier AI training is possible](https://mlsanity.com/paper/2606.05433/) (2026)
- [Multinational AGI Consortium (MAGIC): A Proposal for International Coordination on AI](https://mlsanity.com/paper/2310.09217/) (2023)
- [AI Survival Stories: a Taxonomic Analysis of AI Existential Risk](https://mlsanity.com/paper/2601.09765/) (2026)
- [Privacy-Preserving AI Verification via Minimal Information Disclosure](https://mlsanity.com/paper/2608.02774/) (2026)
- [Workload Identification with Physical Side Channels for AI Governance](https://mlsanity.com/paper/2609.00309/) (2026)
- [Can Power Draw Constrain Covert Compute? Limits of Analogue Verification for AI Governance](https://mlsanity.com/paper/2610.07476/) (2026)

## Abstract (arXiv)

As advanced machine learning systems' capabilities begin to play a significant role in geopolitics and societal order, it may become imperative that (1) governments be able to enforce rules on the development of advanced ML systems within their borders, and (2) countries be able to verify each other's compliance with potential future international agreements on advanced ML development. This work analyzes one mechanism to achieve this, by monitoring the computing hardware used for large-scale NN training. The framework's primary goal is to provide governments high confidence that no actor uses large quantities of specialized ML chips to execute a training run in violation of agreed rules. At the same time, the system does not curtail the use of consumer computing devices, and maintains the privacy and confidentiality of ML practitioners' models, data, and hyperparameters. The system consists of interventions at three stages: (1) using on-chip firmware to occasionally save snapshots of the the neural network weights stored in device memory, in a form that an inspector could later retrieve; (2) saving sufficient information about each training run to prove to inspectors the details of the training run that had resulted in the snapshotted weights; and (3) monitoring the chip supply chain to ensure that no actor can avoid discovery by amassing a large quantity of un-tracked chips. The proposed design decomposes the ML training rule verification problem into a series of narrow technical challenges, including a new variant of the Proof-of-Learning problem [Jia et al. '21].