mlsanityData to 7 Oct 2026

Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?

arXiv 2407.21792 ↗First posted 31 Jul 2024cs.LG cs.AI cs.CLcited by 56

Not on the map. Why?

Abstract (arXiv)

As artificial intelligence systems grow more powerful, there has been increasing interest in "AI safety" research to address emerging and future risks. However, the field of AI safety remains poorly defined and inconsistently measured, leading to confusion about how researchers can contribute. This lack of clarity is compounded by the unclear relationship between AI safety benchmarks and upstream general capabilities (e.g., general knowledge and reasoning). To address these issues, we conduct a comprehensive meta-analysis of AI safety benchmarks, empirically analyzing their correlation with general capabilities across dozens of models and providing a survey of existing directions in AI safety. Our findings reveal that many safety benchmarks highly correlate with both upstream model capabilities and training compute, potentially enabling "safetywashing"--where capability improvements are misrepresented as safety advancements. Based on these findings, we propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context as a set of clearly delineated research goals that are empirically separable from generic capabilities advancements. In doing so, we aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 120 of its 161 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

120 of 161 references.

References · 120

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2023GPT-4 Technical Reportcited by 18,132
  8. 2022Training language models to follow instructions with human feedbackcited by 15,712
  9. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  10. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  12. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
38 more
  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  12. 2017On Calibration of Modern Neural Networkscited by 4,860
  13. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  14. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  15. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  16. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  17. 2022A ConvNet for the 2020scited by 3,545
  18. 2023Qwen Technical Reportcited by 3,273
  19. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  20. 2023Mistral 7Bcited by 3,065
  21. 2016Wide Residual Networkscited by 3,057
  22. 2016Pointer Sentinel Mixture Modelscited by 2,992
  23. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  24. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  25. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  26. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  27. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  28. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  29. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  30. 2022Emergent Abilities of Large Language Modelscited by 2,483
  31. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  32. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  33. 2024Mixtral of Expertscited by 1,785
  34. 2016Concrete Problems in AI Safetycited by 1,582
  35. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  36. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  37. 2019Natural Adversarial Examplescited by 1,401
  38. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330

50 most cited of 120 All 120 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 29; 2026: 22.

56 papers.

Sort:

By area:

  1. 2025Trading Inference-Time Compute for Adversarial Robustnesscited by 60
  2. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
  3. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemscited by 57
  4. 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systemscited by 27
  5. 2025Lessons From Red Teaming 100 Generative AI Productscited by 22
  6. 2024Endless Jailbreaks with Bijection Learningcited by 18
  7. 2025Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Modelscited by 17
  8. 2025SafeVid: Toward Safety Aligned Video Large Multimodal Modelscited by 9
  9. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  10. 2025Benchmarking Misuse Mitigation Against Covert Adversariescited by 8
  11. 2025Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awarenesscited by 7
  12. 2025LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactionscited by 7
38 more
  1. 2026Computational Hermeneutics: Evaluating generative AI as a cultural technologycited by 7
  2. 2024LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistenciescited by 6
  3. 2025Superintelligence Strategy: Expert Versioncited by 6
  4. 2025Bench-2-CoP: Can We Trust Benchmarking for EU AI Compliance?cited by 6
  5. 2024Chat Bankman-Fried: an Exploration of LLM Alignment in Financecited by 5
  6. 2025Sysformer: Safeguarding Frozen Large Language Models with Adaptive System Promptscited by 5
  7. 2025How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulationscited by 5
  8. 2026Breaking Up with Normatively Monolithic Agency with GRACE: A Reason-Based Neuro-Symbolic Architecture for Safe and Ethical AI Alignmentcited by 5
  9. 2025Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learningcited by 4
  10. 2025The Prompt War: How AI Decides on a Military Interventioncited by 4
  11. 2025Psychometric Personality Shaping Modulates Capabilities and Safety in Language Modelscited by 4
  12. 2025Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarkscited by 3
  13. 2025But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectorscited by 3
  14. 2026Evaluating Human-AI Safety: A Framework for Measuring Harmful Capability Upliftcited by 3
  15. 2024Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systemscited by 2
  16. 2025Practical Principles for AI Cost and Compute Accountingcited by 2
  17. 2025MEQA: A Meta-Evaluation Framework for Question & Answer LLM Benchmarkscited by 2
  18. 2025Safety Degradation in AI Agentscited by 2
  19. 2025ConsistencyAI: A Benchmark to Assess LLMs' Factual Consistency When Responding to Different Demographic Groupscited by 2
  20. 2025Mind the Gap! Pathways Towards Unifying AI Safety and Ethics Researchcited by 2
  21. 2026Blind Refusal: Language Models Refuse to Help Users Evade Unjust, Absurd, and Illegitimate Rulescited by 2
  22. 2024Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluationscited by 1
  23. 2025SKATE, a Scalable Tournament Eval: Weaker LLMs differentiate between stronger ones using verifiable challengescited by 1
  24. 2025Safe for Whom? Rethinking How We Evaluate the Safety of LLMs for Real Userscited by 1
  25. 2026Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignmentcited by 1
  26. 2026Do LLMs Follow Their Own Rules? A Reflexive Audit of Self-Stated Safety Policiescited by 1
  27. 2026Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agentscited by 1
  28. 2026Quality Is Not a Safety Proxy Under Quantizationcited by 1
  29. 2026Reinforcement Learning Towards Broadly and Persistently Beneficial Modelscited by 1
  30. 2026Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Modelscited by 1
  31. 2025The bitter lesson of misuse detectioncited by 0
  32. 2025Deprecating Benchmarks: Criteria and Frameworkcited by 0
  33. 2025Get RICH or Die Scaling: Profitably Trading Inference Compute for Robustnesscited by 0
  34. 2026Beyond single-channel agentic benchmarkingcited by 0
  35. 2026Pressure Reveals Character: Behavioural Alignment Evaluation at Depthcited by 0
  36. 2026Incentive Aware AI Regulations: A Credal Characterisationcited by 0
  37. 2026Questionnaire Responses Do not Capture the Safety of AI Agentscited by 0
  38. 2026Safety Training Modulates Harmful Misalignment Under On-Policy RL, But Direction Depends on Environment Designcited by 0

50 most cited of 56 All 56 (JSON)