mlsanityData to 7 Oct 2026

Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation

arXiv 2501.18416 ↗First posted 30 Jan 2025cs.LGcited by 5

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Federated Learning (FL) is increasingly being adopted in military collaborations to develop Large Language Models (LLMs) while preserving data sovereignty. However, prompt injection attacks-malicious manipulations of input prompts-pose new threats that may undermine operational security, disrupt decision-making, and erode trust among allies. This perspective paper highlights four vulnerabilities in federated military LLMs: secret data leakage, free-rider exploitation, system disruption, and misinformation spread. To address these risks, we propose a human-AI collaborative framework with both technical and policy countermeasures. On the technical side, our framework uses red/blue team wargaming and quality assurance to detect and mitigate adversarial behaviors of shared LLM weights. On the policy side, it promotes joint AI-human policy development and verification of security protocols.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 17 of its 29 references that are in the map. First posted on arXiv 30 Jan 2025.

Builds on

17 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Security and Privacy Challenges of Large Language Models: A Survey30 Jan
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 15

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  4. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  5. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  6. 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 91
  7. 2022"Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practicecited by 55
  8. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learningcited by 47
  9. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Modelscited by 40
  10. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  11. 2024The Future of Large Language Model Pre-training is Federatedcited by 30
  12. 2022Federated Learning Challenges and Opportunities: An Outlookcited by 19
3 more
  1. 2024Emerging Safety Attack and Defense in Federated Instruction Tuning of Large Language Modelscited by 18
  2. 2024Neuro-Symbolic AI for Military Applicationscited by 7
  3. 2024Worldwide Federated Training of Language Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2025Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworkscited by 14
  3. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  4. 2025CourtGuard: A Local, Multiagent Prompt Injection Classifiercited by 1
  5. 2026If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systemscited by 1