# Inference-Time Backdoors via Chat Templates: From LLM Supply Chains to Agentic System Compromise

arXiv: https://arxiv.org/abs/2602.04653 · HTML: https://mlsanity.com/paper/2602.04653/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 16 of 16)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training](https://mlsanity.com/paper/2401.05566/) (2024)
- [Defending Against Indirect Prompt Injection Attacks With Spotlighting](https://mlsanity.com/paper/2403.14720/) (2024)
- [Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection](https://mlsanity.com/paper/2307.16888/) (2023)
- [Backdoor Attacks for In-Context Learning with Language Models](https://mlsanity.com/paper/2307.14692/) (2023)
- [Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models](https://mlsanity.com/paper/2305.01219/) (2023)
- [Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning](https://mlsanity.com/paper/2401.05949/) (2024)
- [ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents](https://mlsanity.com/paper/2509.22830/) (2025)
- [ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templates](https://mlsanity.com/paper/2406.12935/) (2024)
- [UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models](https://mlsanity.com/paper/2502.13141/) (2025)
- [A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Models](https://mlsanity.com/paper/2410.04490/) (2024)
- [System Prompt Poisoning: Persistent Attacks on Large Language Models Beyond User Injection](https://mlsanity.com/paper/2505.06493/) (2025)
- [Training-Free Policy Violation Detection via Activation-Space Whitening in LLMs](https://mlsanity.com/paper/2512.03994/) (2025)
- [Steganalysis of AI Models LSB Attacks](https://mlsanity.com/paper/2310.01969/) (2023)

## Built on by (most cited first, 2 of 2)

- [AKRASIA: Stealthy Backdoor Attack on Reasoning-based Code LLMs](https://mlsanity.com/paper/2609.01023/) (2026)
- [What You Sign Is Not What You Run: The Verifiability Gap in Quantized LLM Supply Chains](https://mlsanity.com/paper/2610.19498/) (2026)

## Abstract (arXiv)

Open-weight language models are increasingly used in production settings, raising new security challenges. One prominent threat is backdoor attacks, in which adversaries embed hidden behaviors that activate under specific conditions. Previous work has assumed that adversaries have access to training pipelines or deployment infrastructure. We propose a novel attack surface requiring neither: the "chat template". Chat templates are executable programs invoked at every inference call, often implemented in Jinja2, that occupy a privileged position between user input and model processing. We show that an adversary who distributes a model with a maliciously modified template can implant an inference-time backdoor without modifying model weights, poisoning training data, or controlling runtime infrastructure. We evaluate this attack across three deployment tiers. At the LLM level, triggered backdoors reduce factual accuracy from 90% to 15% on average and induce attacker-controlled URL emission with success rates exceeding 80%, while benign inputs show no measurable degradation; these results hold across eighteen models. At the agent level, template backdoors hijack tool-use across two benchmarks spanning 3,868 episodes, bypassing every tested injection defense offered by the benchmarks while remaining fully dormant absent the trigger. At the multi-agent system level, we demonstrate how a single poisoned artifact compromises a real-world agentic deployment and propagates supply-chain code poisoning downstream. The poisoned artifacts evade all security scans on the largest open model distribution platform; and because the payload is rendered by the template before user input is processed, it is architecturally unreachable by input-level defenses such as prompt injection guardrails. These results establish chat templates as a reliable and undefended attack in the open-weight AI supply chain.