mlsanityData to 14 Oct 2026

Where Do Reasoning Models Refuse?

arXiv 2507.03167 ↗First posted 3 Jul 2025cs.CL cs.AI cs.LGcited by 3

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Chat models without chain-of-thought (CoT) reasoning must decide whether to refuse a harmful request before generating their first response token. Reasoning models, by contrast, produce extended chains of thought before their final output, raising a natural question: where in this process does the decision to refuse occur? We investigate this across four open-source reasoning models. We first show that the CoT causally influences refusal outcomes; fixing a specific reasoning trace substantially reduces variance in whether the model ultimately refuses or complies. Zooming into the reasoning trace, we find that in distilled models, subtle differences in the opening sentence of the CoT can fully determine the model's refusal decision, and that these patterns transfer across models distilled from the same teacher. Finally, we extract linear refusal directions from model activations and show that ablating them increases harmful compliance, though less reliably than the same technique achieves on non-reasoning models, and with non-negligible degradation to general capabilities.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 49 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

30 of 49 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Thought Anchors: Which LLM Reasoning Steps Matter?23 Jun
  2. 2025Reasoning Models Don't Always Say What They Think8 May
  3. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation14 Mar
  4. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithful11 Mar
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar

Other references · 24

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  7. 2024OpenAI o1 System Cardcited by 1,244
  8. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  10. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  11. 2022Toy Models of Superpositioncited by 971
  12. 2023Steering Language Models With Activation Engineeringcited by 922
12 more
  1. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  2. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  3. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  4. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  5. 2024A StrongREJECT for Empty Jailbreakscited by 385
  6. 2024Alignment faking in large language modelscited by 360
  7. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  8. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  9. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  10. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
  11. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 99
  12. 2025Trading Inference-Time Compute for Adversarial Robustnesscited by 61

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Leveraging Intermediate Representations of Time Series Foundation Models for Anomaly Detectioncited by 1
  2. 2026Beyond a Single Direction: Chain-of-Thought Disrupts Simple Steering of Refusalcited by 0
  3. 2026Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
  2. 2026Do Language Models Know When They'll Refuse? Probing Introspective Awareness of Safety Boundaries
  3. 2024Refusal in Language Models Is Mediated by a Single Direction
  4. 2023Measuring Faithfulness in Chain-of-Thought Reasoning
  5. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?
  6. 2024Rethinking harmless refusals when fine-tuning foundation models
  7. 2026Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning
  8. 2026Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models