mlsanityData to 14 Oct 2026

Unveiling Safety Vulnerabilities of Large Language Models

arXiv 2311.04124 ↗First posted 7 Nov 2023cs.CL cs.AI cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

As large language models become more prevalent, their possible harmful or inappropriate responses are a cause for concern. This paper introduces a unique dataset containing adversarial examples in the form of questions, which we call AttaQ, designed to provoke such harmful or inappropriate responses. We assess the efficacy of our dataset by analyzing the vulnerabilities of various models when subjected to it. Additionally, we introduce a novel automatic approach for identifying and naming vulnerable semantic regions - input semantic areas for which the model is likely to produce harmful outputs. This is achieved through the application of specialized clustering techniques that consider both the semantic similarity of the input attacks and the harmfulness of the model's responses. Automatically identifying vulnerable semantic regions enhances the evaluation of model weaknesses, facilitating targeted improvements to its safety mechanisms and overall reliability.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 19 of its 45 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

19 of 45 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  6. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  8. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  9. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  10. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 413
  11. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 232
  12. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 207
7 more
  1. 2020HateBERT: Retraining BERT for Abusive Language Detection in Englishcited by 207
  2. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 185
  3. 2020Recipes for Safety in Open-domain Chatbotscited by 170
  4. 2023MM-LIMA: Less Is More for Alignment in Multi-Modal Datasetscited by 61
  5. 2022Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Modelscited by 49
  6. 2023Visual Instruction Tuning with Polite Flamingocited by 38
  7. 2022Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systemscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 8; 2025: 15; 2026: 3.

27 papers.

Sort:

By area:

  1. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 71
  2. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 68
  3. 2024Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinationscited by 54
  4. 2025Fine-Tuning Lowers Safety and Disrupts Evaluation Consistencycited by 27
  5. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 24
  6. 2024Arabic Dataset for LLM Safeguard Evaluationcited by 17
  7. 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 16
  8. 2025Large Reasoning Models Are Autonomous Jailbreak Agentscited by 13
  9. 2024Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitationscited by 10
  10. 2024Alignment Studio: Aligning Large Language Models to Particular Contextual Regulationscited by 10
  11. 2025Adversarial Prompt Evaluation: Systematic Benchmarking of Guardrails Against Prompt Input Attacks on LLMscited by 9
  12. 2024Large Language Models can be Strong Self-Detoxifierscited by 7
15 more
  1. 2025IF-GUIDE: Influence Function-Guided Detoxification of LLMscited by 7
  2. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 6
  3. 2025Security Steerability is All You Needcited by 4
  4. 2025Dialz: A Python Toolkit for Steering Vectorscited by 4
  5. 2023MetaAID 2.5: A Secure Framework for Developing Metaverse Applications via Large Language Modelscited by 3
  6. 2024Dynamic Normativity: Necessary and Sufficient Conditions for Value Alignmentcited by 3
  7. 2025Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMscited by 3
  8. 2025ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Conceptscited by 2
  9. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Abilitycited by 2
  10. 2026Test-Time Detoxification without Training or Learning Anythingcited by 2
  11. 2025Patching LLM Like Software: A Lightweight Method for Improving Safety Policy in Large Language Modelscited by 1
  12. 2025Adversarial Distilled Retrieval-Augmented Guarding Model for Online Malicious Intent Detectioncited by 0
  13. 2025ASTRA: Agentic Steerability and Risk Assessment Frameworkcited by 0
  14. 2026PsychoSafe: Eliciting Psychologically-Informed Refusals in Large Language Modelscited by 0
  15. 2026Granite.Trust Policy Tools: Shareable, Actionable Policies for Generative AI Applicationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023SC-Safety: A Multi-round Open-ended Question Adversarial Safety Benchmark for Large Language Models in Chinese
  2. 2025Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Models
  3. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
  4. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  5. 2026No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios
  6. 2025Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation
  7. 2025Prompt Injection as an Emerging Threat: Evaluating the Resilience of Large Language Models
  8. 2024Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability