Meta SecAlign: A Secure Foundation LLM Against Prompt Injection Attacks
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Prompt injection attacks, where untrusted data contains an injected prompt to manipulate the system, have been listed as the top security threat to AI agents. By fine-tuning on simulated prompt injections, SecAlign, a leading open defense, reports LLMs with good test-time robustness and negligible benign utility drop. By scaling up training and evaluations, however, we find that SecAlign actually suffers from significant utility degradation, especially in agentic tasks where the threat of prompt injection is prominent. Motivated by this, we propose Meta-SecAlign for utility-preserving defense by (1) randomized injection position during training to avoid shortcut learning and (2) self-generated responses as high-quality in-distribution training labels. Across general knowledge, instruction following, and agentic workflows (on tool-calling and web-navigation), Meta-SecAlign maintains almost all the undefended LLM's utility while achieving better overall security than SecAlign against various static and GCG adaptive attacks. Experiments use Llama-3.1-8B, Llama-3.3-70B, Llama-4-Scout, Qwen3-4B, and Qwen3.6-27B on 6 prompt injection benchmarks including AgentDojo, InjecAgent, WASP, and SEP. Below are links for the code (https://github.com/facebookresearch/Meta_SecAlign), Meta-SecAlign-70B (https://huggingface.co/facebook/Meta-SecAlign-70B), and Meta-SecAlign-8B (https://huggingface.co/facebook/Meta-SecAlign-8B) models.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 34 of its 66 references that are in the map. First posted on arXiv 3 Jul 2025.
Builds on
34 of 66 references.
Closest ancestors · 4
Same direction, newest first
- 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
- 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
- 2025Defeating Prompt Injections by Design24 Mar
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 30
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
18 more
- 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
- 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
- 2019On Evaluating Adversarial Robustnesscited by 447
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 143
- 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 81
- 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 72
- 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 67
- 2025Lessons from Defending Gemini Against Indirect Prompt Injectionscited by 43
- 2025A Critical Evaluation of Defenses against Prompt Injection Attackscited by 36
- 2025A Closer Look at System Prompt Robustnesscited by 32
- 2025Stronger Enforcement of Instruction Hierarchy via Augmented Intermediate Representationscited by 9
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMscited by 17
- 2026StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancingcited by 6
- 2026SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignmentcited by 6
- 2026Know Thy Enemy: Securing LLMs Against Prompt Injection via Diverse Data Synthesis and Instruction-Level Chain-of-Thought Learningcited by 1
- 2026ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflictscited by 1
- 2026Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Trainingcited by 0
- 2026RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agentscited by 0
- 2026AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Modelcited by 0
- 2026NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentscited by 0
- 2026LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defensecited by 0
- 2026LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense8 Oct 2026
- 2026NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents8 Oct 2026
- 2026AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model6 Oct 2026
- 2026RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents5 Oct 2026
- 2026SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment2 Sep 2026
- 2026StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing25 Aug 2026
- 2026Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Training27 May 2026
- 2026ShadowMerge: A Novel Poisoning Attack on Graph-Based Agent Memory via Relation-Channel Conflicts9 May 2026
- 2026Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs25 Mar 2026
- 2026Know Thy Enemy: Securing LLMs Against Prompt Injection via Diverse Data Synthesis and Instruction-Level Chain-of-Thought Learning8 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2024Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications
- 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents
- 2026Defense Against Indirect Prompt Injection via Tool Result Parsing
- 2026Evaluation of Prompt Injection Defenses in Large Language Models
- 2024Enhancing Prompt Injection Attacks to LLMs via Poisoning Alignment
- 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection
- 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
- 2026A Framework for Formalizing LLM Agent Security