mlsanityData to 14 Oct 2026

Meta SecAlign: A Secure Foundation LLM Against Prompt Injection Attacks

arXiv 2507.02735 ↗First posted 3 Jul 2025cs.CR cs.AIcited by 10

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Prompt injection attacks, where untrusted data contains an injected prompt to manipulate the system, have been listed as the top security threat to AI agents. By fine-tuning on simulated prompt injections, SecAlign, a leading open defense, reports LLMs with good test-time robustness and negligible benign utility drop. By scaling up training and evaluations, however, we find that SecAlign actually suffers from significant utility degradation, especially in agentic tasks where the threat of prompt injection is prominent. Motivated by this, we propose Meta-SecAlign for utility-preserving defense by (1) randomized injection position during training to avoid shortcut learning and (2) self-generated responses as high-quality in-distribution training labels. Across general knowledge, instruction following, and agentic workflows (on tool-calling and web-navigation), Meta-SecAlign maintains almost all the undefended LLM's utility while achieving better overall security than SecAlign against various static and GCG adaptive attacks. Experiments use Llama-3.1-8B, Llama-3.3-70B, Llama-4-Scout, Qwen3-4B, and Qwen3.6-27B on 6 prompt injection benchmarks including AgentDojo, InjecAgent, WASP, and SEP. Below are links for the code (https://github.com/facebookresearch/Meta_SecAlign), Meta-SecAlign-70B (https://huggingface.co/facebook/Meta-SecAlign-70B), and Meta-SecAlign-8B (https://huggingface.co/facebook/Meta-SecAlign-8B) models.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 34 of its 66 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

34 of 66 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
  2. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
  3. 2025Defeating Prompt Injections by Design24 Mar
  4. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar

Other references · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  8. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  10. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  11. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
18 more
  1. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  3. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  4. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  5. 2019On Evaluating Adversarial Robustnesscited by 447
  6. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  7. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
  8. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
  9. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
  10. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 210
  11. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 143
  12. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 81
  13. 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 72
  14. 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 67
  15. 2025Lessons from Defending Gemini Against Indirect Prompt Injectionscited by 43
  16. 2025A Critical Evaluation of Defenses against Prompt Injection Attackscited by 36
  17. 2025A Closer Look at System Prompt Robustnesscited by 32
  18. 2025Stronger Enforcement of Instruction Hierarchy via Augmented Intermediate Representationscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications
  2. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents
  3. 2026Defense Against Indirect Prompt Injection via Tool Result Parsing
  4. 2026Evaluation of Prompt Injection Defenses in Large Language Models
  5. 2024Enhancing Prompt Injection Attacks to LLMs via Poisoning Alignment
  6. 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection
  7. 2026Assessing Automated Prompt Injection Attacks in Agentic Environments
  8. 2026A Framework for Formalizing LLM Agent Security