mlsanityData to 14 Oct 2026

HauntAttack: When Attack Follows Reasoning as a Shadow

arXiv 2506.07031 ↗First posted 8 Jun 2025cs.CR cs.AI cs.CLcited by 1

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Emerging Large Reasoning Models (LRMs) consistently excel in mathematical and reasoning tasks, showcasing remarkable capabilities. However, the enhancement of reasoning abilities and the exposure of internal reasoning processes introduce new safety vulnerabilities. A critical question arises: when reasoning becomes intertwined with harmfulness, will LRMs become more vulnerable to jailbreaks in reasoning mode? To investigate this, we introduce HauntAttack, a novel and general-purpose black-box adversarial attack framework that systematically embeds harmful instructions into reasoning questions. Specifically, we modify key reasoning conditions in existing questions with harmful instructions, thereby constructing a reasoning pathway that guides the model step by step toward unsafe outputs. We evaluate HauntAttack on 11 LRMs and observe an average attack success rate of over 70%, achieving up to 13 percentage points of absolute improvement over the strongest prior baseline. Our further analysis reveals that even advanced safety-aligned models remain highly susceptible to reasoning-based attacks, offering insights into the urgent challenge of balancing reasoning capability and safety in future model development.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 32 of its 54 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

32 of 54 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Safety in Large Reasoning Models: A Survey24 Apr
  2. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability14 Apr
  3. 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos19 Feb
  4. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking18 Feb
  5. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities17 Feb
  6. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan

Other references · 26

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  6. 2015The Limitations of Deep Learning in Adversarial Settingscited by 790
  7. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  8. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  10. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  11. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  12. 2023Certifying LLM Safety against Adversarial Promptingcited by 266
14 more
  1. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
  2. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  3. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 230
  4. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
  5. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130
  6. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79
  7. 2024ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectorscited by 51
  8. 2023Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attackscited by 34
  9. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 27
  10. 2023Small Language Models Improve Giants by Rewriting Their Outputscited by 18
  11. 2025TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in Real-World Scenarioscited by 14
  12. 2025LongSafety: Evaluating Long-Context Safety of Large Language Modelscited by 12
  13. 2023Large Language Models Meet Computer Vision: A Brief Surveycited by 5
  14. 2024PizzaCommonSense: Learning to Model Commonsense Reasoning about Intermediate Steps in Cooking Recipescited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Efficiency-Safety Dilemma in Large Reasoning Models
  2. 2026Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models
  3. 2026Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMs
  4. 2024Reasoning as a Weapon: Adaptive Dual-Path Jailbreak Attack on Large Language Models
  5. 2025Weakest Link in the Chain: Security Vulnerabilities in Advanced Reasoning Models
  6. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1
  7. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models
  8. 2026Capability-Routed Guard: Defending Large Reasoning Models Against Reasoning-Centric Jailbreaks