# NeurIPS Should Require Reproducibility Standards for Frontier AI Safety Claims

arXiv: https://arxiv.org/abs/2605.08192 · HTML: https://mlsanity.com/paper/2605.08192/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 5 May 2026.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs](https://mlsanity.com/paper/2406.18495/) (2024)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Frontier AI Regulation: Managing Emerging Risks to Public Safety](https://mlsanity.com/paper/2307.03718/) (2023)
- [Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs](https://mlsanity.com/paper/2509.18058/) (2025)
- [Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming](https://mlsanity.com/paper/2601.18076/) (2026)
- [The 2025 Foundation Model Transparency Index](https://mlsanity.com/paper/2512.10169/) (2025)
- [Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies](https://mlsanity.com/paper/2601.11699/) (2026)
- [Responsible Reporting for Frontier AI Development](https://mlsanity.com/paper/2404.02675/) (2024)
- [STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports](https://mlsanity.com/paper/2508.09853/) (2025)
- [Statistically Valid Post-Deployment Monitoring Should Be Standard for AI-Based Digital Health](https://mlsanity.com/paper/2506.05701/) (2025)
- [Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor](https://mlsanity.com/paper/2506.14652/) (2025)
- [Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track](https://mlsanity.com/paper/2506.19882/) (2025)
- [Position: Ensuring mutual privacy is necessary for effective external evaluation of proprietary AI systems](https://mlsanity.com/paper/2503.01470/) (2025)
- [NeurIPS should lead scientific consensus on AI policy](https://mlsanity.com/paper/2510.00075/) (2025)
- [Evaluating Human-AI Safety: A Framework for Measuring Harmful Capability Uplift](https://mlsanity.com/paper/2603.26676/) (2026)
- [Third-party compliance reviews for frontier AI safety frameworks](https://mlsanity.com/paper/2505.01643/) (2025)

## Abstract (arXiv)

Frontier AI safety claims - published assertions that a highly capable general-purpose model is below a threshold of concern, adequately mitigated, or suitable for release - increasingly shape model deployment, governance, and public trust. Yet the artefacts needed to evaluate them are routinely withheld, producing an evidential inversion: the most consequential claims in AI safety are often the least reproducible. This position paper argues that NeurIPS should require reproducibility standards for papers making such claims, treating non-reproducibility not as a transparency preference but as an evaluation-methodology failure. The 2026 International AI Safety Report [Bengio et al., 2026] concludes that reliable pre-deployment safety testing has become harder to conduct and that models now distinguish test from deployment contexts; the 2025 Foundation Model Transparency Index [Wan et al., 2025] reports a sector-average transparency score of 40/100 with no major developer adequately disclosing train-test overlap; contemporaneous measurement-theory work shows that attack-success-rate comparisons across systems are often founded on low-validity measurements [Chouldechova et al., 2025]. We propose a three-tier disclosure framework, distinguishing public, controlled, and claim-restricted disclosure, paired with a mandatory claim inventory, scope statements, and a phased implementation path with graduated sanctions. The framework treats secrecy and openness as endpoints of a spectrum, with controlled review (via a federated colloquium of qualified secure-review hosts) covering claims whose artefacts cannot be released publicly, and right-scaling claims whose artefacts cannot be reviewed even confidentially. The standard the community applies to its most consequential claims should be at least as high as the standard it applies to its least.