mlsanityData to 14 Oct 2026

AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement

arXiv 2502.16776 ↗First posted 24 Feb 2025cs.CL cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

As AI models are increasingly deployed across diverse real-world scenarios, ensuring their safety remains a critical yet underexplored challenge. While substantial efforts have been made to evaluate and enhance AI safety, the lack of a standardized framework and comprehensive toolkit poses significant obstacles to systematic research and practical adoption. To bridge this gap, we introduce AISafetyLab, a unified framework and toolkit that integrates representative attack, defense, and evaluation methodologies for AI safety. AISafetyLab features an intuitive interface that enables developers to seamlessly apply various techniques while maintaining a well-structured and extensible codebase for future advancements. Additionally, we conduct empirical studies on Vicuna, analyzing different attack and defense strategies to provide valuable insights into their comparative effectiveness. To facilitate ongoing research and development in AI safety, AISafetyLab is publicly available at https://github.com/thu-coai/AISafetyLab, and we are committed to its continuous maintenance and improvement.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

34 of 47 references.

References · 34

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  3. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  7. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  8. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  10. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
  11. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  12. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
22 more
  1. 2023Detecting Language Model Attacks with Perplexitycited by 431
  2. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  3. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  4. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  5. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  6. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
  7. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 306
  8. 2023Certifying LLM Safety against Adversarial Promptingcited by 266
  9. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
  10. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
  11. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  12. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  13. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  14. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 199
  15. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 192
  16. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
  17. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 157
  18. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 79
  19. 2024ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectorscited by 51
  20. 2023Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimationcited by 36
  21. 2024From Theft to Bomb-Making: The Ripple Effect of Unlearning in Defending Against Jailbreak Attackscited by 29
  22. 2024Self-Evaluation as a Defense Against Adversarial Attacks on LLMscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 4.

12 papers.

  1. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 26
  2. 2025RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguardscited by 14
  3. 2025TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluationscited by 7
  4. 2025Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!cited by 5
  5. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attackscited by 4
  6. 2025RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaborationcited by 4
  7. 2025Silent Tokens, Loud Effects: Padding in LLMscited by 4
  8. 2026The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoningcited by 3
  9. 2026SoK: Robustness in Large Language Models against Jailbreak Attackscited by 3
  10. 2026GUARD-SLM: Token Activation-Based Defense Against Jailbreak Attacks for Small Language Modelscited by 1
  11. 2025EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Modelscited by 0
  12. 2026Few-Shot Truly Benign DPO Attack for Jailbreaking LLMscited by 0