AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations
Not on the map. Why?
Abstract (arXiv)
The rapid advancement of AI systems has raised widespread concerns about potential harms of frontier AI systems and the need for responsible evaluation and oversight. In this position paper, we argue that frontier AI companies should report both pre- and post-mitigation safety evaluations to enable informed policy decisions. Evaluating models at both stages provides policymakers with essential evidence to regulate deployment, access, and safety standards. We show that relying on either in isolation can create a misleading picture of model safety. Our analysis of AI safety disclosures from leading frontier labs identifies three critical gaps: (1) companies rarely evaluate both pre- and post-mitigation versions, (2) evaluation methods lack standardization, and (3) reported results are often too vague to inform policy. To address these issues, we recommend mandatory disclosure of pre- and post-mitigation capabilities to approved government bodies, standardized evaluation methods, and minimum transparency requirements for public safety reporting. These ensure that policymakers and regulators can craft targeted safety measures, assess deployment risks, and scrutinize companies' safety claims effectively.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 17 Mar 2025.
Builds on
24 of 46 references.
References · 24
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024GPT-4o System Cardcited by 4,467
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
- 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
12 more
- 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
- 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
- 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
- 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
- 2024Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptationcited by 74
- 2024Safety Cases: How to Justify the Safety of Advanced AI Systemscited by 48
- 2023BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13Bcited by 35
- 2023Exploiting Novel GPT-4 APIscited by 27
- 2024Scaling Trends for Data Poisoning in LLMscited by 26
- 2024Badllama 3: removing safety finetuning from Llama 3 in minutescited by 10
- 2024What AI evaluations for preventing catastrophic risks can and cannot docited by 4
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.