# Why and How Governments Should Monitor AI Development

arXiv: https://arxiv.org/abs/2108.12427 · HTML: https://mlsanity.com/paper/2108.12427/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 23 references that are in the map.
- First posted on arXiv 28 Aug 2021.

## Builds on (most cited first, 7 of 7)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation](https://mlsanity.com/paper/1802.07228/) (2018)
- [The AI Index 2021 Annual Report](https://mlsanity.com/paper/2103.06312/) (2021)

## Built on by (most cited first, 14 of 14)

- [Model evaluation for extreme risks](https://mlsanity.com/paper/2305.15324/) (2023)
- [Frontier AI Regulation: Managing Emerging Risks to Public Safety](https://mlsanity.com/paper/2307.03718/) (2023)
- [Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework](https://mlsanity.com/paper/2311.14711/) (2023)
- [Responsible Reporting for Frontier AI Development](https://mlsanity.com/paper/2404.02675/) (2024)
- [Trust AI Regulation? Discerning users are vital to build trust and effective AI regulation](https://mlsanity.com/paper/2403.09510/) (2024)
- [An International Consortium for Evaluations of Societal-Scale Risks from Advanced AI](https://mlsanity.com/paper/2310.14455/) (2023)
- [Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting](https://mlsanity.com/paper/2606.09809/) (2026)
- [Negative Human Rights as a Basis for Long-term AI Safety and Regulation](https://mlsanity.com/paper/2208.14788/) (2022)
- [Both eyes open: Vigilant Incentives help Regulatory Markets improve AI Safety](https://mlsanity.com/paper/2303.03174/) (2023)
- [Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations](https://mlsanity.com/paper/2412.15433/) (2024)
- [One Bad NOFO? AI Governance in Federal Grantmaking](https://mlsanity.com/paper/2505.08133/) (2025)
- [Risks of AI-driven product development and strategies for their mitigation](https://mlsanity.com/paper/2506.00047/) (2025)
- [Artificial Intelligence and Arms Control](https://mlsanity.com/paper/2211.00065/) (2022)
- [Fairness: from the ethical principle to the practice of Machine Learning development as an ongoing agreement with stakeholders](https://mlsanity.com/paper/2304.06031/) (2023)

## Abstract (arXiv)

In this paper we outline a proposal for improving the governance of artificial intelligence (AI) by investing in government capacity to systematically measure and monitor the capabilities and impacts of AI systems. If adopted, this would give governments greater information about the AI ecosystem, equipping them to more effectively direct AI development and deployment in the most societally and economically beneficial directions. It would also create infrastructure that could rapidly identify potential threats or harms that could occur as a consequence of changes in the AI ecosystem, such as the emergence of strategically transformative capabilities, or the deployment of harmful systems. We begin by outlining the problem which motivates this proposal: in brief, traditional governance approaches struggle to keep pace with the speed of progress in AI. We then present our proposal for addressing this problem: governments must invest in measurement and monitoring infrastructure. We discuss this proposal in detail, outlining what specific things governments could focus on measuring and monitoring, and the kinds of benefits this would generate for policymaking. Finally, we outline some potential pilot projects and some considerations for implementing this in practice.