mlsanityData to 7 Oct 2026

Conversational Complexity for Assessing Risk in Large Language Models

arXiv 2409.01247 ↗First posted 2 Sep 2024cs.AI cs.CL cs.ITcited by 6

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Large Language Models (LLMs) present a dual-use dilemma: they enable beneficial applications while harboring potential for harm, particularly through conversational interactions. Despite various safeguards, advanced LLMs remain vulnerable. A watershed case in early 2023 involved journalist Kevin Roose's extended dialogue with Bing, an LLM-powered search engine, which revealed harmful outputs after probing questions, highlighting vulnerabilities in the model's safeguards. This contrasts with simpler early jailbreaks, like the "Grandma Jailbreak," where users framed requests as innocent help for a grandmother, easily eliciting similar content. This raises the question: How much conversational effort is needed to elicit harmful information from LLMs? We propose two measures to quantify this effort: Conversational Length (CL), which measures the number of conversational turns needed to obtain a specific harmful response, and Conversational Complexity (CC), defined as the Kolmogorov complexity of the user's instruction sequence leading to the harmful response. To address the incomputability of Kolmogorov complexity, we approximate CC using a reference LLM to estimate the compressibility of the user instructions. Applying this approach to a large red-teaming dataset, we perform a quantitative analysis examining the statistical distribution of harmful and harmless conversational lengths and complexities. Our empirical findings suggest that this distributional analysis and the minimization of CC serve as valuable tools for understanding AI safety, offering insights into the accessibility of harmful information. This work establishes a foundation for a new perspective on LLM safety, centered around the algorithmic complexity of pathways to harm.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 27 of its 62 references that are in the map. First posted on arXiv 2 Sep 2024.

Builds on

27 of 62 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  6. 2022Red Teaming Language Models with Language Modelscited by 983
  7. 2021Ethical and social risks of harm from Language Modelscited by 957
  8. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  9. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  10. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  11. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  12. 2023Are aligned neural networks adversarially aligned?cited by 305
15 more
  1. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  2. 2023Role-Play with Large Language Modelscited by 290
  3. 2023Language Modeling Is Compressioncited by 225
  4. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  5. 2022Predictability and Surprise in Large Generative Modelscited by 209
  6. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  7. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 172
  8. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 121
  9. 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Modelscited by 100
  10. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 94
  11. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  12. 2022Complementary Explanations for Effective In-Context Learningcited by 71
  13. 2023Red Teaming Language Model Detectors with Language Modelscited by 55
  14. 2023LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?cited by 47
  15. 2024From "um" to "yeah": Producing, predicting, and regulating information flow in human conversationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2026Seven simple steps for log analysis in AI systemscited by 5
  2. 2025Supervision policies can shape long-term risk management in general-purpose AI modelscited by 2
  3. 2025How Catastrophic is Your LLM? Certifying Risk in Conversationcited by 2
  4. 2025Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreakingcited by 0
  5. 2026Fundamental Limitation in Explaining AIcited by 0
  6. 2026To Reason or to Fabricate: Reasoning Without Shortcuts via Hint-Anchored Pairwise Aggregationcited by 0