mlsanityData to 7 Oct 2026

Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking

arXiv 2311.09827 ↗First posted 16 Nov 2023cs.CLcited by 78

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

While large language models (LLMs) have demonstrated increasing power, they have also given rise to a wide range of harmful behaviors. As representatives, jailbreak attacks can provoke harmful or unethical responses from LLMs, even after safety alignment. In this paper, we investigate a novel category of jailbreak attacks specifically designed to target the cognitive structure and processes of LLMs. Specifically, we analyze the safety vulnerability of LLMs in the face of (1) multilingual cognitive overload, (2) veiled expression, and (3) effect-to-cause reasoning. Different from previous jailbreak attacks, our proposed cognitive overload is a black-box attack with no need for knowledge of model architecture or access to model weights. Experiments conducted on AdvBench and MasterKey reveal that various LLMs, including both popular open-source model Llama 2 and the proprietary model ChatGPT, can be compromised through cognitive overload. Motivated by cognitive psychology work on managing cognitive load, we further investigate defending cognitive overload attack from two perspectives. Empirical studies show that our cognitive overload from three perspectives can jailbreak all studied LLMs successfully, while existing defense strategies can hardly mitigate the caused malicious uses effectively.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 34 of its 64 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

34 of 64 references.

References · 34

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  11. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  12. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
22 more
  1. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  2. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  3. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  4. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  5. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  6. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  7. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  8. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  9. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  10. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  11. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  12. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  13. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  14. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  15. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  16. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  17. 2022Predictability and Surprise in Large Generative Modelscited by 209
  18. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  19. 2023Large Language Models as Zero-Shot Conversational Recommenderscited by 126
  20. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  21. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 56
  22. 2023Building Real-World Meeting Summarization Systems using Large Language Models: A Practical Perspectivecited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 35; 2025: 29; 2026: 14.

78 papers.

Sort:

By area:

  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2024A StrongREJECT for Empty Jailbreakscited by 352
  3. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  4. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  5. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  6. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  7. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  8. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 79
  9. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  10. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  11. 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
  12. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 44
38 more
  1. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  2. 2024Understanding the Dark Side of LLMs' Intrinsic Self-Correctioncited by 42
  3. 2024Distract Large Language Models for Automatic Jailbreak Attackcited by 40
  4. 2024Boosting Jailbreak Attack with Momentumcited by 38
  5. 2024Leveraging the Context through Multi-Round Interactions for Jailbreaking Attackscited by 32
  6. 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 32
  7. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Modelscited by 31
  8. 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queriescited by 26
  9. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 22
  10. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  11. 2024Sandwich attack: Multi-language Mixture Adaptive Attack on LLMscited by 20
  12. 2024Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Coursecited by 19
  13. 2025MPO: Multilingual Safety Alignment via Reward Gap Optimizationcited by 18
  14. 2024The Sound of Healthcare: Improving Medical Transcription ASR Accuracy with Large Language Modelscited by 17
  15. 2024Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Modelscited by 15
  16. 2024Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Modelscited by 15
  17. 2024Mitigating Backdoor Threats to Large Language Models: Advancement and Challengescited by 14
  18. 2025Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Controlcited by 14
  19. 2024Cognitive Overload Attack:Prompt Injection for Long Contextcited by 13
  20. 2025E2AT: Multimodal Jailbreak Defense via Dynamic Joint Optimization for Multimodal Large Language Modelscited by 12
  21. 2025Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behaviorcited by 12
  22. 2024SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkagecited by 10
  23. 2024JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMscited by 10
  24. 2025JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steeringcited by 10
  25. 2025Jailbreaking on Text-to-Video Models via Scene Splitting Strategycited by 10
  26. 2024Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerationscited by 8
  27. 2025MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attackscited by 8
  28. 2025Activation Steering for Masked Diffusion Language Modelscited by 7
  29. 2025Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyondcited by 6
  30. 2025Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretabilitycited by 6
  31. 2025ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistantscited by 6
  32. 2025Bypassing Prompt Guards in Production with Controlled-Release Promptingcited by 6
  33. 2024Enhancing the Capability and Robustness of Large Language Models through Reinforcement Learning-Driven Query Refinementcited by 5
  34. 2025A Review of Incorporating Psychological Theories in LLMscited by 5
  35. 2024Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learningcited by 4
  36. 2025On Surjectivity of Neural Networks: Can you elicit any behavior from your model?cited by 4
  37. 2025How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversationscited by 3
  38. 2026Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Awaycited by 3

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025The Cost of Thinking: Increased Jailbreak Risk in Large Language Models
  2. 2025Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
  3. 2025SafeBehavior: Simulating Human-Like Multistage Reasoning to Mitigate Jailbreak Attacks in Large Language Models
  4. 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?
  5. 2024Defending against Jailbreak through Early Exit Generation of Large Language Models
  6. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models
  7. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  8. 2025Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions