mlsanityData to 7 Oct 2026

PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models

arXiv 2310.12439 ↗First posted 19 Oct 2023cs.CL cs.AIcited by 40

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

Prompts have significantly improved the performance of pretrained Large Language Models (LLMs) on various downstream tasks recently, making them increasingly indispensable for a diverse range of LLM application scenarios. However, the backdoor vulnerability, a serious security threat that can maliciously alter the victim model's normal predictions, has not been sufficiently explored for prompt-based LLMs. In this paper, we present POISONPROMPT, a novel backdoor attack capable of successfully compromising both hard and soft prompt-based LLMs. We evaluate the effectiveness, fidelity, and robustness of POISONPROMPT through extensive experiments on three popular prompt methods, using six datasets and three widely used LLMs. Our findings highlight the potential security threats posed by backdoor attacks on prompt-based LLMs and emphasize the need for further research in this area.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 14 of its 19 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

14 of 19 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models28 May
  2. 2022BadPrompt: Backdoor Attacks on Continuous Prompts27 Nov

Other references · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2021GPT Understands, Toocited by 715
  7. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  8. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  9. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  10. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  11. 2022Exploring the Universal Vulnerability of Prompt-based Learning Paradigmcited by 56
  12. 2023PromptCARE: Prompt Copyright Protection by Watermark Injection and Verificationcited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 15; 2025: 20; 2026: 3.

40 papers.

Sort:

By area:

  1. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  3. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 94
  4. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  5. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Modelscited by 63
  6. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
  7. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  8. 2024Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attributioncited by 35
  9. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  10. 2024CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularizationcited by 29
  11. 2024Scaling Trends for Data Poisoning in LLMscited by 26
  12. 2025Data Poisoning in Deep Learning: A Surveycited by 20
28 more
  1. 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Modelscited by 13
  2. 2025Attack and defense techniques in large language models: A survey and new perspectivescited by 13
  3. 2025A Survey of Attacks on Large Language Modelscited by 13
  4. 2025Erased but Not Forgotten: How Backdoors Compromise Concept Erasurecited by 9
  5. 2025MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systemscited by 8
  6. 2025Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigationcited by 5
  7. 2023Survey on AI Ethics: A Socio-technical Perspectivecited by 4
  8. 2024vTune: Verifiable Fine-Tuning for LLMs Through Backdooringcited by 4
  9. 2024Neutralizing Backdoors through Information Conflicts for Large Language Modelscited by 4
  10. 2025Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMscited by 4
  11. 2025Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustnesscited by 4
  12. 2025PoisonedParrot: Subtle Data Poisoning Attacks to Elicit Copyright-Infringing Content from Large Language Modelscited by 3
  13. 2025Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Modelscited by 3
  14. 2026Architecture Matters for Multi-Agent Securitycited by 3
  15. 2024New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlookcited by 2
  16. 2025Evaluating the efficacy of LLM Safety Solutions : The Palit Benchmark Datasetcited by 2
  17. 2025A Linear Approach to Data Poisoningcited by 2
  18. 2025BadReward: Clean-Label Poisoning of Reward Models in Text-to-Image RLHFcited by 2
  19. 2025Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessmentcited by 2
  20. 2024Mitigating Adversarial Attacks in LLMs through Defensive Suffix Generationcited by 1
  21. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  22. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  23. 2025Key-Conditioned Orthonormal Transform Gating (K-OTG): Multi-Key Access Control with Hidden-State Scrambling for LoRA-Tuned Modelscited by 1
  24. 2026Stealthy Backdoor Attacks against LLMs Based on Natural Style Triggerscited by 1
  25. 2024Shortcuts Arising from Contrast: Effective and Covert Clean-Label Attacks in Prompt-Based Learningcited by 0
  26. 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
  27. 2025Provably Secure Retrieval-Augmented Generationcited by 0
  28. 2026Trojaning the Alignment: Stealthy Backdoor Attacks against Graph Foundation Modelscited by 0