mlsanityData to 7 Oct 2026

Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection

arXiv 2609.35932 ↗First posted 28 Sep 2026cs.CR cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Prompt injection against LLM agents becomes much stronger when the injected instruction is wrapped in the model's own chat template. A forged template marker such as <|im_start|> can reach the model either as a single reserved control token or as a sequence of ordinary subword tokens. The two decode to exactly the same text, and because tokenization runs on the server, the defender rather than the attacker decides which one the model receives. We use this to measure how much of the injected instruction's authority comes from the reserved token's learned representation. Encoding the forged markers as subwords, with the text held fixed and a control for the extra tokens this adds, lowers attack success on the InjecAgent benchmark by 39 to 66 percentage points on three of four open-weight families, and the gap carries over to multi-turn agent tasks in AgentDojo. On Qwen3-8B the gap is 8 points, because without reserved ids the model still recognises the forged turn from its text by reasoning; suppressing the reasoning block widens the gap to 50. The authority sits in the single learned vector at the marker position: the mean of the marker's subword vectors does not reproduce it, the vector of the nearest ordinary token restores the attack on Llama-3.1, and an adaptive attacker who searches for non-reserved markers finds such embedding neighbours on three of four families. In every base and instruction-tuned pair we test, instruction tuning strengthens the model's preference for reserved markers. The standard mitigation, a tokenizer option that encodes special tokens as ordinary subwords, applies only to tokens a configuration declares special, so in 33 of 67 distinct tokenizer configurations, covering 255 of the 400 most-downloaded chat models on Hugging Face, it leaves intact the tool-protocol tokens through which agents read untrusted tool output, and the gap persists on that channel.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 22 of its 35 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

22 of 35 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Automating Agent Hijacking via Structural Template Injection18 Feb
  2. 2025ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents26 Sep

Other references · 20

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  4. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  5. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  6. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  7. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  8. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
  9. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  10. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  11. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  12. 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 62
8 more
  1. 2025A Critical Evaluation of Defenses against Prompt Injection Attackscited by 31
  2. 2024ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templatescited by 29
  3. 2025ASIDE: Architectural Separation of Instructions and Data in Language Modelscited by 21
  4. 2024Virtual Context: Enhancing Jailbreak Attacks with Special Token Injectioncited by 16
  5. 2025Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizationscited by 16
  6. 2025MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulationcited by 2
  7. 2025Separator Injection Attack: Uncovering Dialogue Biases in Large Language Models Caused by Role Separatorscited by 1
  8. 2025TokenBreak: Bypassing Text Classification Models Through Token Manipulationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Control-Token Injection Suppresses Chain-of-Thought and Defeats Reasoning-Based Oversight in Tool-Using Agents
  2. 2026AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories
  3. 2026Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systems
  4. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
  5. 2026CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering
  6. 2026Defusing Explosive Prompts: Understanding and Preventing Trigger-Based Prompt Injections in LLM Agents
  7. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents
  8. 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection