mlsanityData to 14 Oct 2026

Chain-of-Thought Driven Adversarial Scenario Extrapolation for Robust Language Models

arXiv 2505.17089 ↗First posted 20 May 2025cs.CLcited by 4

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) exhibit impressive capabilities, but remain susceptible to a growing spectrum of safety risks, including jailbreaks, toxic content, hallucinations, and bias. Existing defenses often address only a single threat type or resort to rigid outright rejection, sacrificing user experience and failing to generalize across diverse and novel attacks. This paper introduces Adversarial Scenario Extrapolation (ASE), a novel inference-time computation framework that leverages Chain-of-Thought (CoT) reasoning to simultaneously enhance LLM robustness and seamlessness. ASE guides the LLM through a self-generative process of contemplating potential adversarial scenarios and formulating defensive strategies before generating a response to the user query. Comprehensive evaluation on four adversarial benchmarks with four latest LLMs shows that ASE achieves near-zero jailbreak attack success rates and minimal toxicity, while slashing outright rejections to <4%. ASE outperforms six state-of-the-art defenses in robustness-seamlessness trade-offs, with 92-99% accuracy on adversarial Q&A and 4-10x lower bias scores. By transforming adversarial perception into an intrinsic cognitive process, ASE sets a new paradigm for secure and natural human-AI interaction.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 71 of its 93 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

71 of 93 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking18 Feb

Other references · 70

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  7. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  10. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  11. 2021Ethical and social risks of harm from Language Modelscited by 966
  12. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
38 more
  1. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  2. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  3. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  4. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  5. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
  6. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  7. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  9. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
  10. 2023Detecting Language Model Attacks with Perplexitycited by 431
  11. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  12. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  13. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  14. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  15. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  16. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  17. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  18. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  19. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  20. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  21. 2023Certifying LLM Safety against Adversarial Promptingcited by 266
  22. 2022Prompting GPT-3 To Be Reliablecited by 260
  23. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
  24. 2023Challenges and Applications of Large Language Modelscited by 254
  25. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  26. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
  27. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 195
  28. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 192
  29. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
  30. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  31. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
  32. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 172
  33. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 171
  34. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
  35. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
  36. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 157
  37. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 124
  38. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 114

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research
  2. 2024Towards Assurance of LLM Adversarial Robustness using Ontology-Driven Argumentation
  3. 2025Adversarial Reasoning at Jailbreaking Time
  4. 2023Robustness Over Time: Understanding Adversarial Examples' Effectiveness on Longitudinal Versions of Large Language Models
  5. 2024Enhancing Adversarial Attacks through Chain of Thought
  6. 2025Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generation
  7. 2025Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment
  8. 2023Jailbreaker in Jail: Moving Target Defense for Large Language Models