mlsanityData to 14 Oct 2026

Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming

arXiv 2601.18076 ↗First posted 26 Jan 2026cs.LGcited by 15

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

We argue that conclusions drawn about relative system safety or attack method efficacy via AI red teaming are often not supported by evidence provided by attack success rate (ASR) comparisons. We show, through conceptual, theoretical, and empirical contributions, that many conclusions are founded on apples-to-oranges comparisons or low-validity measurements. Our arguments are grounded in asking a simple question: When can attack success rates be meaningfully compared? To answer this question, we draw on ideas from social science measurement theory and inferential statistics, which, taken together, provide a conceptual grounding for understanding when numerical values obtained through the quantification of system attributes can be meaningfully compared. Through this lens, we articulate conditions under which ASRs can and cannot be meaningfully compared. Using jailbreaking as a running example, we provide examples and extensive discussion of apples-to-oranges ASR comparisons and measurement validity challenges.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 37 of its 50 references that are in the map. First posted on arXiv 26 Jan 2026.

Builds on

37 of 50 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025OpenAI's Approach to External Red Teaming for AI Models and Systems24 Jan
  2. 2024Diverse and Effective Red Teaming with Auto-generated Rewards and Multi-step Reinforcement Learning24 Dec
  3. 2024Against The Achilles' Heel: A Survey on Red Teaming for Generative Models31 Mar
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal6 Feb
  5. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?29 Jan
  6. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratch15 Jun
  7. 2022Red Teaming Language Models with Language Models7 Feb

Other references · 30

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  5. 2024A Survey on LLM-as-a-Judgecited by 1,423
  6. 2021Membership Inference Attacks From First Principlescited by 878
  7. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  9. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  10. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  11. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  12. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
18 more
  1. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
  2. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  3. 2019Measurement and Fairnesscited by 308
  4. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  5. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  6. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 160
  7. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
  8. 2024Best-of-N Jailbreakingcited by 76
  9. 2025Position: Evaluating Generative AI Systems Is a Social Science Measurement Challengecited by 63
  10. 2025From Hard Refusals to Safe-Completions: Toward Output-Centric Safety Trainingcited by 52
  11. 2023Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chaincited by 46
  12. 2024Measuring memorization in language models via probabilistic extractioncited by 41
  13. 2025Extracting memorized pieces of (copyrighted) books from open-weight language modelscited by 30
  14. 2024Position: Measure Dataset Diversity, Don't Just Claim Itcited by 26
  15. 2024"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreakcited by 25
  16. 2023Evaluating General-Purpose AI with Psychometricscited by 23
  17. 2021Hyperparameter Optimization Is Deceiving Us, and How to Stop Itcited by 20
  18. 2024A Shared Standard for Valid Measurement of Generative AI Systems' Capabilities, Risks, and Impactscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

  1. 2026REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systemscited by 4
  2. 2026Offline Preference-Based Trajectory Evaluationcited by 2
  3. 2026When No Benchmark Exists: Validating Comparative LLM Safety Scoring Without Ground-Truth Labelscited by 1
  4. 2026CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluationcited by 1
  5. 2026NeurIPS Should Require Reproducibility Standards for Frontier AI Safety Claimscited by 0
  6. 2026Why Do Safety Guardrails Degrade Across Languages?cited by 0
  7. 2026PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Modelscited by 0
  8. 2026Extractable Memorization From First Principlescited by 0
  9. 2026What AI Red-Team Evaluations Can and Cannot Provecited by 0
  10. 2026STEMMA: An Adversarial Multi-Agent Framework for Evaluating Self-Identity Consistency in LLMscited by 0
  11. 2026An Investigation of the NeurIPS and ICML 2025 Position Trackscited by 0
  12. 2026Validity-Aware Jailbreak Evaluation for Large Language Modelscited by 0
3 more
  1. 2026Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdictscited by 0
  2. 2026Divide, Consult, Conquer: Capability Laundering Through Aligned LLMscited by 0
  3. 2026False Floors: LLM Safety Routing Evaluations Break Under Distribution Shiftcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation
  2. 2025The Automation Advantage in AI Red Teaming
  3. 2026A Translational Note on AI Safety Evaluation
  4. 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming
  5. 2026Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents
  6. 2025Red Teaming AI Red Teaming
  7. 2025From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teaming
  8. 2024The Human Factor in AI Red Teaming: Perspectives from Social and Collaborative Computing