Chain-of-Thought Driven Adversarial Scenario Extrapolation for Robust Language Models
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Large Language Models (LLMs) exhibit impressive capabilities, but remain susceptible to a growing spectrum of safety risks, including jailbreaks, toxic content, hallucinations, and bias. Existing defenses often address only a single threat type or resort to rigid outright rejection, sacrificing user experience and failing to generalize across diverse and novel attacks. This paper introduces Adversarial Scenario Extrapolation (ASE), a novel inference-time computation framework that leverages Chain-of-Thought (CoT) reasoning to simultaneously enhance LLM robustness and seamlessness. ASE guides the LLM through a self-generative process of contemplating potential adversarial scenarios and formulating defensive strategies before generating a response to the user query. Comprehensive evaluation on four adversarial benchmarks with four latest LLMs shows that ASE achieves near-zero jailbreak attack success rates and minimal toxicity, while slashing outright rejections to <4%. ASE outperforms six state-of-the-art defenses in robustness-seamlessness trade-offs, with 92-99% accuracy on adversarial Q&A and 4-10x lower bias scores. By transforming adversarial perception into an intrinsic cognitive process, ASE sets a new paradigm for secure and natural human-AI interaction.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 71 of its 93 references that are in the map. First posted on arXiv 20 May 2025.
Builds on
71 of 93 references.
Closest ancestors · 1
Same direction, newest first
- 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking18 Feb
Other references · 70
- 2023GPT-4 Technical Reportcited by 18,396
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
38 more
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
- 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
- 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
- 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
- 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
- 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
- 2023Detecting Language Model Attacks with Perplexitycited by 431
- 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
- 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
- 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
- 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
- 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
- 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
- 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
- 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
- 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
- 2023Certifying LLM Safety against Adversarial Promptingcited by 266
- 2022Prompting GPT-3 To Be Reliablecited by 260
- 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
- 2023Challenges and Applications of Large Language Modelscited by 254
- 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
- 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
- 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 195
- 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 192
- 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
- 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
- 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
- 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 172
- 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 171
- 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
- 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
- 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 157
- 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 124
- 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 114
50 most cited of 70 All 70 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 3.
4 papers.
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 8
- 2026REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systemscited by 4
- 2026Robustness of Vision Language Models Against Split-Image Harmful Input Attackscited by 1
- 2026Directional Embedding Smoothing for Robust Vision Language Modelscited by 0
- 2026REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems11 Aug 2026
- 2026Directional Embedding Smoothing for Robust Vision Language Models16 Mar 2026
- 2026Robustness of Vision Language Models Against Split-Image Harmful Input Attacks8 Feb 2026
- 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models13 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research
- 2024Towards Assurance of LLM Adversarial Robustness using Ontology-Driven Argumentation
- 2025Adversarial Reasoning at Jailbreaking Time
- 2023Robustness Over Time: Understanding Adversarial Examples' Effectiveness on Longitudinal Versions of Large Language Models
- 2024Enhancing Adversarial Attacks through Chain of Thought
- 2025Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generation
- 2025Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment
- 2023Jailbreaker in Jail: Moving Target Defense for Large Language Models