mlsanityData to 7 Oct 2026

Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey

arXiv 2402.09283 ↗First posted 14 Feb 2024cs.CL cs.AI cs.CYcited by 124

Not on the map. Why?

Cited by 59 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large Language Models (LLMs) are now commonplace in conversation applications. However, their risks of misuse for generating harmful responses have raised serious societal concerns and spurred recent research on LLM conversation safety. Therefore, in this survey, we provide a comprehensive overview of recent studies, covering three critical aspects of LLM conversation safety: attacks, defenses, and evaluations. Our goal is to provide a structured summary that enhances understanding of LLM conversation safety and encourages further investigation into this important subject. For easy reference, we have categorized all the studies mentioned in this survey according to our taxonomy, available at: https://github.com/niconi19/LLM-conversation-safety.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). Builds on 76 of its 100 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

76 of 100 references.

References · 76

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  10. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  11. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  12. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
38 more
  1. 2023LIMA: Less Is More for Alignmentcited by 1,080
  2. 2022Red Teaming Language Models with Language Modelscited by 983
  3. 2021Ethical and social risks of harm from Language Modelscited by 957
  4. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  5. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  6. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  7. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  8. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  9. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  10. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  11. 2019Defending Against Neural Fake Newscited by 553
  12. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  13. 2023Detecting Language Model Attacks with Perplexitycited by 411
  14. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  15. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  16. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  17. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  18. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  19. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  20. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  21. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  22. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  23. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  24. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  25. 2023Poisoning Language Models During Instruction Tuningcited by 254
  26. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  27. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  28. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  29. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  30. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  31. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  32. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  33. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  34. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  35. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 161
  36. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
  37. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  38. 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacycited by 137

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 33; 2025: 61; 2026: 30.

124 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 280
  3. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  4. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  5. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 88
  6. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  7. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  8. 2025G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systemscited by 56
  9. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  10. 2024Language Models Resist Alignment: Evidence From Data Compressioncited by 50
  11. 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMscited by 48
  12. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
38 more
  1. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
  2. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
  3. 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 36
  4. 2025AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Managementcited by 35
  5. 2024Immunization against harmful fine-tuning attackscited by 33
  6. 2024Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentationcited by 24
  7. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  8. 2025Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimizationcited by 23
  9. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?cited by 22
  10. 2024When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?cited by 20
  11. 2025Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Modelscited by 19
  12. 2024MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usabilitycited by 14
  13. 2024CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Modelscited by 13
  14. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 13
  15. 2025BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judgecited by 13
  16. 2025Collab: Controlled Decoding using Mixture of Agents for LLM Alignmentcited by 13
  17. 2025UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Modelscited by 12
  18. 2025ControlNET: A Firewall for RAG-based LLM Systemcited by 11
  19. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11
  20. 2024Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Toolscited by 9
  21. 2025A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Modelscited by 9
  22. 2024Risks, Causes, and Mitigations of Widespread Deployments of Large Language Models (LLMs): A Surveycited by 8
  23. 2025Automated Capability Discovery via Foundation Model Self-Explorationcited by 8
  24. 2025Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancementcited by 8
  25. 2025Measuring Harmfulness of Computer-Using Agentscited by 8
  26. 2025Sequences of Logits Reveal the Low Rank Structure of Language Modelscited by 8
  27. 2025Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detectioncited by 8
  28. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instancecited by 8
  29. 2024LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Modelscited by 7
  30. 2024SMILES-Prompting: A Novel Approach to LLM Jailbreak Attacks in Chemical Synthesiscited by 7
  31. 2024LLM-Virus: Evolutionary Jailbreak Attack on Large Language Modelscited by 7
  32. 2025FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Modelscited by 7
  33. 2026Understanding and Preserving Safety in Fine-Tuned LLMscited by 7
  34. 2024ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Modelscited by 6
  35. 2024Vulnerability of LLMs to Vertically Aligned Text Manipulationscited by 6
  36. 2025From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMscited by 6
  37. 2025Should LLM Safety Be More Than Refusing Harmful Instructions?cited by 6
  38. 2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignmentcited by 6

50 most cited of 124 All 124 (JSON)