Do Reasoning LLMs Refuse What They Infer in Long Contexts?
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
Long-context LLMs can infer objectives that are not stated explicitly. This capability is useful for reasoning over documents, code, retrieved evidence, and tool traces, but it also creates a safety risk: harmful intent can be distributed across a context and become visible only after the model composes the relevant pieces. Existing safety evaluations mostly test explicit harmful requests, and therefore miss this failure mode. We introduce compositional reasoning attacks, a long-context threat model in which harmful requests are decomposed into semantically incomplete fragments and embedded in long contexts. The final query is neutral; the harmful objective emerges only if the model retrieves the fragments, composes them, and infers the implied goal. We instantiate this setting using AdvBench requests, varying the required reasoning from Direct Retrieval to Single-hop Aggregation, Chain Reasoning, and Multi-hop Deductive Reasoning, and evaluate 15 frontier LLMs on contexts up to 64k tokens. Models usually refuse harmful requests when they are directly retrievable. However, refusal rates drop sharply when the same objectives must be reconstructed compositionally, often with larger failures in longer contexts. Benign reconstruction and fragment-position analyses indicate that these failures are not mainly retrieval errors: models often infer the harmful objective and then comply. Increasing inference-time reasoning improves refusal but remains incomplete and costly. Our results reveal a long-context safety gap: current models are better at refusing harmful requests they see than harmful objectives they infer.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 25 of its 38 references that are in the map. First posted on arXiv 9 Feb 2026.
Builds on
25 of 38 references.
Closest ancestors · 10
Same direction, newest first
- 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Models19 Nov
- 2025Chain-of-Thought Hijacking30 Oct
- 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models24 Oct
- 2025ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models22 May
- 2025AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Models16 May
- 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos19 Feb
- 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking18 Feb
- 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities17 Feb
- 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models16 Feb
- 2025Adversarial Reasoning at Jailbreaking Time3 Feb
Other references · 15
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
- 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
- 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 120
- 2025A Comprehensive Survey on Long Context Language Modelingcited by 113
- 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
- 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakerscited by 91
- 2025Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Modelscited by 58
- 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansioncited by 47
- 2024NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densitiescited by 32
- 2024FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Modelscited by 29
- 2024MathHay: An Automated Benchmark for Long-Context Mathematical Reasoning in LLMscited by 13
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026A Decomposition Perspective to Long-context Reasoning for LLMs
- 2026Positional Failures in Long-Context LLMs: A Blind Spot in Reasoning Benchmarks
- 2025When Refusals Fail: Unstable Safety Mechanisms in Long-Context LLM Agents
- 2026Know Your Limits : On the Faithfulness of LLMs as Solvers and Autoformalizers in Legal Reasoning
- 2026First Token Matters: Understanding Safety Collapse in Large Reasoning Models
- 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs
- 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety
- 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning