Soft Instruction De-escalation Defense
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Large Language Models (LLMs) are increasingly deployed in agentic systems that interact with an external environment; this makes them susceptible to prompt injections when dealing with untrusted data. To overcome this limitation, we propose SIC (Soft Instruction Control)-a simple yet effective iterative prompt sanitization loop designed for tool-augmented LLM agents. Our method repeatedly inspects incoming data for instructions that could compromise agent behavior. If such content is found, the malicious content is rewritten, masked, or removed, and the result is re-evaluated. The process continues until the input is clean or a maximum iteration limit is reached; if imperative instruction-like content remains, the agent halts to ensure security. By allowing multiple passes, our approach acknowledges that individual rewrites may fail but enables the system to catch and correct missed injections in later steps. Although immediately useful, worst-case analysis shows that SIC is not infallible; strong adversary can still get a 15% ASR by embedding non-imperative workflows. This nonetheless raises the bar.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 20 of its 36 references that are in the map. First posted on arXiv 24 Oct 2025.
Builds on
20 of 36 references.
Closest ancestors · 6
Same direction, newest first
- 2025PromptArmor: Simple yet Effective Prompt Injection Defenses21 Jul
- 2025Progent: Securing AI Agents with Privilege Control16 Apr
- 2025Defeating Prompt Injections by Design24 Mar
- 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
- 2024Imprompter: Tricking LLM Agents into Improper Tool Use19 Oct
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 14
- 2024GPT-4o System Cardcited by 4,467
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
- 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
- 2022TALM: Tool Augmented Language Modelscited by 197
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
- 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 91
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Evaluation and Hardening of LLM System Instructions Against Extraction via Encoding Attacks
- 2025CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitization
- 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation
- 2025SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection
- 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection
- 2026Silent Egress: When Implicit Prompt Injection Makes LLM Agents Leak Without a Trace
- 2025Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysis