mlsanityData to 7 Oct 2026

CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference

arXiv 2406.17626 ↗First posted 25 Jun 2024cs.CL cs.AIcited by 51

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

As large language models (LLMs) constantly evolve, ensuring their safety remains a critical research problem. Previous red-teaming approaches for LLM safety have primarily focused on single prompt attacks or goal hijacking. To the best of our knowledge, we are the first to study LLM safety in multi-turn dialogue coreference. We created a dataset of 1,400 questions across 14 categories, each featuring multi-turn coreference safety attacks. We then conducted detailed evaluations on five widely used open-source LLMs. The results indicated that under multi-turn coreference safety attacks, the highest attack success rate was 56% with the LLaMA2-Chat-7b model, while the lowest was 13.9% with the Mistral-7B-Instruct model. These findings highlight the safety vulnerabilities in LLMs during dialogue coreference interactions.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 19 of its 28 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

19 of 28 references.

References · 19

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  2. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  3. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  4. 2023Whose Opinions Do Language Models Reflect?cited by 759
  5. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  6. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  7. 2018Gender Bias in Coreference Resolutioncited by 438
  8. 2017End-to-end Neural Coreference Resolutioncited by 240
  9. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  10. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
  11. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  12. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
7 more
  1. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  2. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  3. 2022When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgmentcited by 93
  4. 2016Deep Reinforcement Learning for Mention-Ranking Coreference Modelscited by 63
  5. 2022Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Modelscited by 49
  6. 2021SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failurescited by 37
  7. 2022Coreference Resolution through a seq2seq Transition-Based Systemcited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 27; 2026: 20.

51 papers.

Sort:

By area:

  1. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
  2. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  3. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  4. 2024FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMscited by 36
  5. 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teamingcited by 31
  6. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  7. 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
  8. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  9. 2025Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attackscited by 20
  10. 2025Human Decision-making is Susceptible to AI-driven Manipulationcited by 19
  11. 2026Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluationcited by 15
  12. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 14
38 more
  1. 2024UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Modelscited by 13
  2. 2025Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Studycited by 12
  3. 2025Evaluating the Sensitivity of LLMs to Prior Contextcited by 9
  4. 2026DeepContext: Stateful Real-Time Detection of Multi-Turn Adversarial Intent Drift in LLMscited by 8
  5. 2025AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelismcited by 7
  6. 2025RedDebate: Safer Responses Through Multi-Agent Red Teaming Debatescited by 7
  7. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safetycited by 7
  8. 2025Dagger Behind Smile: Fool LLMs with a Happy Ending Storycited by 5
  9. 2025MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?cited by 5
  10. 2025Jailbreak Distillation: Renewable Safety Benchmarkingcited by 4
  11. 2025OffTopicEval: When Large Language Models Enter the Wrong Chat, Almost Always!cited by 4
  12. 2025SafeMT: Multi-turn Safety for Multimodal Language Modelscited by 4
  13. 2025Guarding the Guardrails: A Taxonomy-Driven Approach to Jailbreak Detectioncited by 4
  14. 2025ObjexMT: Objective Extraction and Metacognitive Calibration for LLM-as-a-Judge under Multi-Turn Jailbreakscited by 3
  15. 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligencecited by 3
  16. 2026RedBench: A Universal Dataset for Comprehensive Red Teaming of Large Language Modelscited by 3
  17. 2026Improving the Safety and Trustworthiness of Medical AI via Multi-Agent Evaluation Loopscited by 3
  18. 2026SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systemscited by 3
  19. 2024RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreakingcited by 2
  20. 2025Jailbreaking LLMs via Semantically Relevant Nested Scenarios with Targeted Toxic Knowledgecited by 2
  21. 2025How Catastrophic is Your LLM? Certifying Risk in Conversationcited by 2
  22. 2026Investigating and Alleviating Harm Amplification in LLM Interactionscited by 2
  23. 2025Overview of Dialog System Evaluation Track: Dimensionality, Language, Culture and Safety at DSTC 12cited by 1
  24. 2025Bidirectional Intention Inference Enhances LLMs' Defense Against Multi-Turn Jailbreak Attackscited by 1
  25. 2025LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Modelscited by 1
  26. 2025Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attackscited by 1
  27. 2026Lattice: Generative Guardrails for Conversational Agentscited by 1
  28. 2026AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Modelscited by 1
  29. 2026Defensive M2S: Training Guardrail Models on Compressed Multi-turn Conversationscited by 0
  30. 2026Evaluation of Audio Language Models for Fairness, Safety, and Securitycited by 0
  31. 2026EthicMind: A Risk-Aware Framework for Ethical-Emotional Alignment in Multi-Turn Dialoguecited by 0
  32. 2026SaFeR-Steer: Evolving Multi-Turn MLLMs via Synthetic Bootstrapping and Feedback Dynamicscited by 0
  33. 2026RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generationcited by 0
  34. 2026The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysiscited by 0
  35. 2026The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systemscited by 0
  36. 2026Stateful Guardrails for Multi-Turn LLM Systems: A Conversational Risk Accumulation Frameworkcited by 0
  37. 2026Before the Script, Set the Stage: How Worldview Simulation Amplifies Psychologically Grounded Persuasion in Multi-Turn Jailbreakingcited by 0
  38. 2026DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactionscited by 0

50 most cited of 51 All 51 (JSON)