# Threats to Pre-trained Language Models: Survey and Taxonomy

arXiv: https://arxiv.org/abs/2202.06862 · HTML: https://mlsanity.com/paper/2202.06862/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2022.

## Built on by (most cited first, 19 of 19)

- [Large Language Model Alignment: A Survey](https://mlsanity.com/paper/2309.15025/) (2023)
- [Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey](https://mlsanity.com/paper/2302.10035/) (2023)
- [CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks](https://mlsanity.com/paper/2209.08773/) (2022)
- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?](https://mlsanity.com/paper/2308.01990/) (2023)
- [Citation: A Key to Building Responsible and Accountable Large Language Models](https://mlsanity.com/paper/2307.02185/) (2023)
- [METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities](https://mlsanity.com/paper/2312.06056/) (2023)
- [Challenges and Remedies to Privacy and Security in AIGC: Exploring the Potential of Privacy Computing, Blockchain, and Beyond](https://mlsanity.com/paper/2306.00419/) (2023)
- [Backdoor Attacks with Input-unique Triggers in NLP](https://mlsanity.com/paper/2303.14325/) (2023)
- [A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models](https://mlsanity.com/paper/2307.12980/) (2023)
- [Metamorphic Testing of Large Language Models for Natural Language Processing](https://mlsanity.com/paper/2511.02108/) (2025)
- [ChatGPT-Like Large-Scale Foundation Models for Prognostics and Health Management: A Survey and Roadmaps](https://mlsanity.com/paper/2305.06472/) (2023)
- [UOR: Universal Backdoor Attacks on Pre-trained Language Models](https://mlsanity.com/paper/2305.09574/) (2023)
- [How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical Survey](https://mlsanity.com/paper/2312.04775/) (2023)
- [Understanding Impacts of Task Similarity on Backdoor Attack and Detection](https://mlsanity.com/paper/2210.06509/) (2022)
- [PeaTMOSS: Mining Pre-Trained Models in Open-Source Software](https://mlsanity.com/paper/2310.03620/) (2023)
- [LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters](https://mlsanity.com/paper/2405.16287/) (2024)
- [The Ripple Effect: On Unforeseen Complications of Backdoor Attacks](https://mlsanity.com/paper/2505.11586/) (2025)
- [LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems](https://mlsanity.com/paper/2509.10682/) (2025)

## Abstract (arXiv)

Pre-trained language models (PTLMs) have achieved great success and remarkable performance over a wide range of natural language processing (NLP) tasks. However, there are also growing concerns regarding the potential security issues in the adoption of PTLMs. In this survey, we comprehensively systematize recently discovered threats to PTLM systems and applications. We perform our attack characterization from three interesting perspectives. (1) We show threats can occur at different stages of the PTLM pipeline raised by different malicious entities. (2) We identify two types of model transferability (landscape, portrait) that facilitate attacks. (3) Based on the attack goals, we summarize four categories of attacks (backdoor, evasion, data privacy and model privacy). We also discuss some open problems and research directions. We believe our survey and taxonomy will inspire future studies towards secure and privacy-preserving PTLMs.