mlsanityData to 14 Oct 2026

AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research

arXiv 2511.04316 ↗First posted 6 Nov 2025cs.AI cs.SEcited by 6

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

The rapid expansion of research on Large Language Model (LLM) safety and robustness has produced a fragmented and oftentimes buggy ecosystem of implementations, datasets, and evaluation methods. This fragmentation makes reproducibility and comparability across studies challenging, hindering meaningful progress. To address these issues, we introduce AdversariaLLM, a toolbox for conducting LLM jailbreak robustness research. Its design centers on reproducibility, correctness, and extensibility. The framework implements twelve adversarial attack algorithms, integrates seven benchmark datasets spanning harmfulness, over-refusal, and utility evaluation, and provides access to a wide range of open-weight LLMs via Hugging Face. The implementation includes advanced features for comparability and reproducibility such as compute-resource tracking, deterministic results, and distributional evaluation techniques. name also integrates judging through the companion package JudgeZoo, which can also be used independently. Together, these components aim to establish a robust foundation for transparent, comparable, and reproducible research in LLM safety.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 39 of its 51 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

39 of 51 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs6 Mar
  2. 2025REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective24 Feb
  3. 2024AdvPrefix: An Objective for Nuanced LLM Jailbreaks13 Dec
  4. 2024Best-of-N Jailbreaking4 Dec
  5. 2024An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks21 Oct
  6. 2024JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework11 Oct
  7. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models13 Jun
  8. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs11 Apr
  9. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  10. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack2 Apr
  11. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  12. 2024Fast Adversarial Attacks on Language Models In One GPU Minute23 Feb
8 more
  1. 2024A StrongREJECT for Empty Jailbreaks15 Feb
  2. 2024Attacking Large Language Models with Projected Gradient Descent14 Feb
  3. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space14 Feb
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  5. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  7. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 19

  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  3. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  4. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
  5. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  6. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  7. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  8. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 136
  9. 2016Technical Report on the CleverHans v2.1.0 Adversarial Examples Librarycited by 124
  10. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
  11. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  12. 2024garak: A Framework for Security Probing Large Language Modelscited by 49
7 more
  1. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Modelscited by 35
  2. 2024A Probabilistic Perspective on Unlearning and Alignment for Large Language Modelscited by 21
  3. 2025LLM-Safety Evaluations Lack Robustnesscited by 18
  4. 2025Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Modelscited by 17
  5. 2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 15
  6. 2025Sampling-aware Adversarial Attacks Against Large Language Modelscited by 9
  7. 2025Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectivescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustnesscited by 20
  2. 2026Closing the Distribution Gap in Adversarial Training for LLMscited by 0
  3. 2026Test-Time Training Undermines Safety Guardrailscited by 0
  4. 2026All you need to break LLMs are Black-Box, Adapting, Efficient, Transferable, Harmful, Applicable ... Attackscited by 0
  5. 2026REALM: A Unified Red-Teaming Benchmark for Physical-World VLMscited by 0
  6. 2026Exploring Jailbreak Vulnerability Across Depth-Based Dynamic Inference in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Robustness Over Time: Understanding Adversarial Examples' Effectiveness on Longitudinal Versions of Large Language Models
  2. 2026SoK: Robustness in Large Language Models against Jailbreak Attacks
  3. 2025LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge
  4. 2024Robustness of Large Language Models Against Adversarial Attacks
  5. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge
  6. 2024Large Language Models as Robust Data Generators in Software Analytics: Are We There Yet?
  7. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  8. 2025Inverse Language Modeling towards Robust and Grounded LLMs