mlsanityData to 14 Oct 2026

GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methods

arXiv 2502.16903 ↗First posted 24 Feb 2025cs.CL cs.CRcited by 12

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Despite the growing interest in jailbreaks as an effective red-teaming tool for building safe and responsible large language models (LLMs), flawed evaluation system designs have led to significant discrepancies in their effectiveness assessments. With a systematic measurement study based on 37 jailbreak studies since 2022, we find that existing evaluation systems lack case-specific criteria, resulting in misleading conclusions about their effectiveness and safety implications. In this paper, we introduce GuidedBench, a novel benchmark comprising a curated harmful question dataset and GuidedEval, an evaluation system integrated with detailed case-by-case evaluation guidelines. Experiments demonstrate that GuidedBench offers more accurate evaluations of jailbreak performance, enabling meaningful comparisons across methods. GuidedEval reduces inter-evaluator variance by at least 76.03%, ensuring reliable and reproducible evaluations. We reveal why existing jailbreak benchmarks fail to evaluate accurately and suggest better evaluation practices.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 61 of its 86 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

61 of 86 references.

Closest ancestors · 38

Same direction, newest first

  1. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs3 Oct
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey5 Jul
  3. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models26 Jun
  4. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs13 Jun
  5. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses3 Jun
  6. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs21 Apr
  7. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs11 Apr
  8. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  9. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  10. 2024Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models26 Mar
  11. 2024Distract Large Language Models for Automatic Jailbreak Attack13 Mar
  12. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction28 Feb
26 more
  1. 2024CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models26 Feb
  2. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers25 Feb
  3. 2024ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings25 Feb
  4. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails24 Feb
  5. 2024Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMs21 Feb
  6. 2024Query-Based Adversarial Prompt Generation19 Feb
  7. 2024When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers16 Feb
  8. 2024A StrongREJECT for Empty Jailbreaks15 Feb
  9. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models5 Feb
  10. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  11. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  12. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation6 Nov
  13. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  14. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks16 Oct
  15. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  16. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  17. 2023Multilingual Jailbreak Challenges in Large Language Models10 Oct
  18. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  19. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  20. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  21. 2023FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models11 Sep
  22. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Models4 Sep
  23. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  24. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  25. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  26. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul

Other references · 23

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  5. 2024A Survey on LLM-as-a-Judgecited by 1,423
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  8. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  9. 2023Prediction-Powered Inferencecited by 220
  10. 2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 168
  11. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 128
  12. 2024MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmarkcited by 105
11 more
  1. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 102
  2. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
  3. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 82
  4. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 76
  5. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 76
  6. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 67
  7. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 50
  8. 2023Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Modelscited by 39
  9. 2023Goal-Oriented Prompt Attack and Safety Evaluation for LLMscited by 24
  10. 2025HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaignscited by 14
  11. 2024SAGE-RT: Synthetic Alignment data Generation for Safety Evaluation and Red Teamingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 11.

12 papers.

  1. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safetycited by 8
  2. 2025TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluationscited by 7
  3. 2026Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarkingcited by 2
  4. 2026JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Modelscited by 2
  5. 2026When Scanners Lie: Evaluator Instability in LLM Red-Teamingcited by 1
  6. 2026CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluationcited by 1
  7. 2026Validity-Aware Jailbreak Evaluation for Large Language Modelscited by 0
  8. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0
  9. 2026Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdictscited by 0
  10. 2026VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformationcited by 0
  11. 2026Can Agents Trust Their Skills? Uncovering Unsafe Chains of Trust in Skill-Based LLM Agentscited by 0
  12. 2026Does Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive Encoderscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models
  2. 2024Rethinking How to Evaluate Language Model Jailbreak
  3. 2024Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks
  4. 2025JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoring
  5. 2024AttackEval: How to Evaluate the Effectiveness of Jailbreak Attacking on Large Language Models
  6. 2026How Real is Your Jailbreak? Fine-grained Jailbreak Evaluation with Anchored Reference
  7. 2024JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models
  8. 2026Did the LLM Answer the Question? A Semantic Reconstruction Game for Jailbreak Evaluation