mlsanityData to 14 Oct 2026

TRACER: Trace-Regularized Training Against Indirect Prompt Injection

arXiv 2610.19133 ↗First posted 13 Oct 2026cs.LGcited by 0

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Indirect prompt injection uses untrusted retrieved content, tool outputs, or agent messages to override an LLM's intended behavior. We show that defense robustness is channel-conditioned: the same attack succeeds 18% of the time as a follow-up message but 53% when placed in retrieved context. We propose TRACER (Trace-Regularized Adversarial Channel Exposure with Reward-shaping), a channel-aware adversarial training method that samples presentation wrappers and combines an answer-anchored, evidence-gated reward with ATRC, which aligns attacked reasoning with clean evidence when an injection is invalid. On GSM8K with Qwen2.5-3B-Instruct, TRACER reduces RAG-framed ASR from 65.5% to 6.1% while preserving 78-82% clean accuracy. Joint RAG+tool training achieves 13.9% average ASR across framings at 3B scale, while a single epoch at 7B reaches 9.4%. These results demonstrate that robust defenses must explicitly train and evaluate across deployment channels.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 33 of its 48 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

33 of 48 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations11 Mar
  2. 2026ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction24 Feb
  3. 2026Defense Against Indirect Prompt Injection via Tool Result Parsing8 Jan
  4. 2025TopicAttack: An Indirect Prompt Injection Attack via Topic Transition18 Jul
  5. 2025A Critical Evaluation of Defenses against Prompt Injection Attacks23 May
  6. 2025Defending against Indirect Prompt Injection by Instruction Detection8 May
  7. 2025Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction29 Apr
  8. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  9. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
  10. 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
  11. 2024SecAlign: Defending Against Prompt Injection with Preference Optimization7 Oct
  12. 2024Defending Against Indirect Prompt Injection Attacks With Spotlighting20 Mar
1 more
  1. 2024StruQ: Defending Against Prompt Injection with Structured Queries9 Feb

Other references · 20

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  10. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  11. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  12. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
8 more
  1. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  2. 2025Reasoning with Exploration: An Entropy Perspectivecited by 281
  3. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 95
  4. 2025Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problemscited by 87
  5. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 86
  6. 2025Outcome-based Exploration for LLM Reasoningcited by 83
  7. 2026Hidden-in-Plain-Text: A Benchmark for Social-Web Indirect Prompt Injection in RAGcited by 5
  8. 2026CORE: Collaborative Reasoning via Cross Teachingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering
  2. 2026Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment
  3. 2026Render Before Reading: Visual Rendering as a Prompt Injection Defense
  4. 2026Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Training
  5. 2026RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents
  6. 2026CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement
  7. 2026TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation
  8. 2025CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancement