# Pitfalls of Evidence-Based AI Policy

arXiv: https://arxiv.org/abs/2502.09618 · HTML: https://mlsanity.com/paper/2502.09618/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2025.

## Built on by (most cited first, 11 of 11)

- [Measuring and mitigating overreliance to build human-compatible AI](https://mlsanity.com/paper/2509.08010/) (2025)
- [The Singapore Consensus on Global AI Safety Research Priorities](https://mlsanity.com/paper/2506.20702/) (2025)
- [Position: Anthropomorphic Misalignment Research Needs Stronger Evidence](https://mlsanity.com/paper/2606.07612/) (2026)
- [A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i](https://mlsanity.com/paper/2505.00808/) (2025)
- [NeurIPS should lead scientific consensus on AI policy](https://mlsanity.com/paper/2510.00075/) (2025)
- [Internal Deployment Gaps in AI Regulation](https://mlsanity.com/paper/2601.08005/) (2026)
- [CALMA: A Process for Deriving Context-aligned Axes for Language Model Alignment](https://mlsanity.com/paper/2507.09060/) (2025)
- [The Stories We Govern By: AI, Risk, and the Power of Imaginaries](https://mlsanity.com/paper/2508.11729/) (2025)
- [Why They Disagree: Decoding Differences in Opinions about AI Risk](https://mlsanity.com/paper/2512.06350/) (2025)
- [AI Cap-and-Trade: Efficiency Incentives for Accessibility and Sustainability](https://mlsanity.com/paper/2601.19886/) (2026)
- [Visible to the Court: How AI Is (and Isn't) Litigated in U.S. Federal Court Opinions](https://mlsanity.com/paper/2607.23888/) (2026)

## Abstract (arXiv)

Nations across the world are working to govern AI. However, from a technical perspective, there is uncertainty and disagreement on the best way to do this. Meanwhile, recent debates over AI regulation have led to calls for "evidence-based AI policy" which emphasize holding regulatory action to a high evidentiary standard. Evidence is of irreplaceable value to policymaking. However, holding regulatory action to too high an evidentiary standard can lead to systematic neglect of certain risks. In historical policy debates (e.g., over tobacco ca. 1965 and fossil fuels ca. 1985) "evidence-based policy" rhetoric is also a well-precedented strategy to downplay the urgency of action, delay regulation, and protect industry interests. Here, we argue that if the goal is evidence-based AI policy, the first regulatory objective must be to actively facilitate the process of identifying, studying, and deliberating about AI risks. We discuss a set of 15 regulatory goals to facilitate this and show that Brazil, Canada, China, the EU, South Korea, the UK, and the USA all have substantial opportunities to adopt further evidence-seeking policies.