# Reasons to Doubt the Impact of AI Risk Evaluations

arXiv: https://arxiv.org/abs/2408.02565 · HTML: https://mlsanity.com/paper/2408.02565/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2024.

## Built on by (most cited first, 4 of 4)

- [Robust ML Auditing using Prior Knowledge](https://mlsanity.com/paper/2505.04796/) (2025)
- [Declare and Justify: Explicit assumptions in AI evaluations are necessary for effective regulation](https://mlsanity.com/paper/2411.12820/) (2024)
- [The Prompt War: How AI Decides on a Military Intervention](https://mlsanity.com/paper/2507.06277/) (2025)
- [The Closing Window: How Governments Could Lose Their Ability to Restrain Advanced AI](https://mlsanity.com/paper/2608.05173/) (2026)

## Abstract (arXiv)

AI safety practitioners invest considerable resources in AI system evaluations, but these investments may be wasted if evaluations fail to realize their impact. This paper questions the core value proposition of evaluations: that they significantly improve our understanding of AI risks and, consequently, our ability to mitigate those risks. Evaluations may fail to improve understanding in six ways, such as risks manifesting beyond the AI system or insignificant returns from evaluations compared to real-world observations. Improved understanding may also not lead to better risk mitigation in four ways, including challenges in upholding and enforcing commitments. Evaluations could even be harmful, for example, by triggering the weaponization of dual-use capabilities or invoking high opportunity costs for AI safety. This paper concludes with considerations for improving evaluation practices and 12 recommendations for AI labs, external evaluators, regulators, and academic researchers to encourage a more strategic and impactful approach to AI risk assessment and mitigation.