Control-Token Injection Suppresses Chain-of-Thought and Defeats Reasoning-Based Oversight in Tool-Using Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The safety of a tool-using language model agent is usually treated as a property of the model alone. We give controlled, full-precision evidence that it is instead a joint property of the model and the software that renders its chat template and parses its tool calls, the decoding harness, and that both halves are attackable from untrusted input. On the released gpt-oss-20b reasoning model under its published tool sandbox, appending a single string of the model's own channel-control tokens to a user message makes the tokenizer render a reasoning turn that is already complete, so the model writes no chain-of-thought and proceeds directly to the tool call. Across forty tasks the model already completes, the reasoning channel falls from a mean of 52.5 tokens to zero on every trial while the http.post still fires on every trial. A rule monitor and a cross-family language-model monitor detect the unsafe request on all plain trials and no forged trials, and on overtly malicious requests the attack converts 39.6% of the model's refusals into completed exfiltrations. Separately, whether an identical tool-call generation fires is decided by the harness parser, not the model: a truncation-tolerant regular expression fires a call whose closing token is missing while a strict one drops it, and two parsers shipped for the Gemma agent give opposite outcomes on identical greedy generations, firing on all twenty-four trials and on none. We show the suppression can be delivered indirectly and characterize its dependence on the chat template across two more reasoning models, and we evaluate input sanitization, parser hardening, and empty-reasoning detection as defenses; flagging an absent trace catches the basic attack but not an adaptive benign decoy. All measurements use greedy decoding on publicly released models. Code and per-trial logs: https://github.com/Usama1002/deleting-the-trace
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 17 of its 22 references that are in the map. First posted on arXiv 23 Sep 2026.
Builds on
17 of 22 references.
Closest ancestors · 7
Same direction, newest first
- 2026Automating Agent Hijacking via Structural Template Injection18 Feb
- 2025ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents26 Sep
- 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents21 Aug
- 2025PromptArmor: Simple yet Effective Prompt Injection Defenses21 Jul
- 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
- 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 10
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
- 2025When Chain of Thought is Necessary, Language Models Struggle to Evade Monitorscited by 88
- 2026Reasoning Models Struggle to Control their Chains of Thoughtcited by 25
- 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbositycited by 17
- 2025Investigating CoT Monitorability in Large Reasoning Modelscited by 9
- 2025Quant Fever, Reasoning Blackholes, Schrodinger's Compliance, and More: Probing GPT-OSS-20Bcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
- 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection
- 2026Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection
- 2026Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
- 2026Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents
- 2026Same Payload, Different Channel: Measuring Trust Asymmetry in Tool-Using Language Models
- 2026Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models
- 2026Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety