mlsanityData to 13 Oct 2026

A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly

arXiv 2312.02003 ↗First posted 4 Dec 2023cs.CR cs.AIcited by 387

Not on the map. Why?

Cited by 95 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large Language Models (LLMs), such as ChatGPT and Bard, have revolutionized natural language understanding and generation. They possess deep language comprehension, human-like text generation capabilities, contextual awareness, and robust problem-solving skills, making them invaluable in various domains (e.g., search engines, customer support, translation). In the meantime, LLMs have also gained traction in the security community, revealing security vulnerabilities and showcasing their potential in security-related tasks. This paper explores the intersection of LLMs with security and privacy. Specifically, we investigate how LLMs positively impact security and privacy, potential risks and threats associated with their use, and inherent vulnerabilities within LLMs. Through a comprehensive literature review, the paper categorizes the papers into "The Good" (beneficial LLM applications), "The Bad" (offensive applications), and "The Ugly" (vulnerabilities of LLMs and their defenses). We have some interesting findings. For example, LLMs have proven to enhance code security (code vulnerability detection) and data privacy (data confidentiality protection), outperforming traditional methods. However, they can also be harnessed for various attacks (particularly user-level attacks) due to their human-like reasoning abilities. We have identified areas that require further research efforts. For example, Research on model and parameter extraction attacks is limited and often theoretical, hindered by LLM parameter scale and confidentiality. Safe instruction tuning, a recent development, requires more exploration. We hope that our work can shed light on the LLMs' potential to both bolster and jeopardize cybersecurity.

Key facts. Cited by 387 papers in the map (data through 13 Oct 2026). Builds on 201 of its 342 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

201 of 342 references.

References · 201

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2023GPT-4 Technical Reportcited by 18,278
  4. 2022Training language models to follow instructions with human feedbackcited by 15,856
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,379
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,064
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,949
  10. 2023A Survey of Large Language Modelscited by 2,559
  11. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,273
  12. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,978
38 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,823
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,637
  3. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,249
  4. 2023A Survey on Evaluation of Large Language Modelscited by 1,236
  5. 2021Ethical and social risks of harm from Language Modelscited by 964
  6. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 960
  7. 2021Membership Inference Attacks From First Principlescited by 872
  8. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 777
  9. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 760
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 710
  11. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 647
  12. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 556
  13. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 485
  14. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 464
  15. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 459
  16. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 429
  17. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  18. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  19. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  20. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 395
  21. 2020Label-Only Membership Inference Attackscited by 386
  22. 2022CodeT: Code Generation with Generated Testscited by 377
  23. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
  24. 2016FastText.zip: Compressing text classification modelscited by 333
  25. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  26. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  27. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  28. 2023Poisoning Language Models During Instruction Tuningcited by 265
  29. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 263
  30. 2020ONION: A Simple and Effective Defense Against Textual Backdoor Attackscited by 262
  31. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  32. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 243
  33. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 235
  34. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 224
  35. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  36. 2020Information Leakage in Embedding Modelscited by 203
  37. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  38. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 198

50 most cited of 201 All 201 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 3; 2024: 135; 2025: 182; 2026: 67.

387 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 625
  2. 2025Why Do Multi-Agent LLM Systems Fail?cited by 573
  3. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 253
  4. 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 241
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 229
  6. 2024Tool Learning with Large Language Models: A Surveycited by 221
  7. 2024A Survey on Efficient Inference for Large Language Modelscited by 202
  8. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 185
  9. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  10. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 128
  11. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 127
  12. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 125
38 more
  1. 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluationscited by 118
  2. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 116
  3. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 112
  4. 2025Baichuan-Omni-1.5 Technical Reportcited by 108
  5. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 106
  6. 2024On-Device Language Models: A Comprehensive Reviewcited by 102
  7. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 95
  8. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 90
  9. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 89
  10. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 87
  11. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 83
  12. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agentscited by 79
  13. 2024WildVision: Evaluating Vision-Language Models in the Wild with Human Preferencescited by 76
  14. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 74
  15. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 71
  16. 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 69
  17. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 68
  18. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 67
  19. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 67
  20. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  21. 2024(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advicecited by 63
  22. 2025Authenticated Delegation and Authorized AI Agentscited by 63
  23. 2024Instruction Backdoor Attacks Against Customized LLMscited by 61
  24. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyondcited by 60
  25. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 59
  26. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  27. 2024Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classificationcited by 53
  28. 2024Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Reviewcited by 51
  29. 2024Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologiescited by 47
  30. 2024Octopus v2: On-device language model for super agentcited by 45
  31. 2025LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insightscited by 44
  32. 2024Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characterscited by 42
  33. 2024D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Modelscited by 41
  34. 2024Boosting Jailbreak Attack with Momentumcited by 40
  35. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
  36. 2024HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testingcited by 36
  37. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  38. 2024Large Language Models for UAVs: Current State and Pathways to the Futurecited by 33

50 most cited of 387 All 387 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey
  2. 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation
  3. 2025A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy
  4. 2025The Model's Language Matters: A Comparative Privacy Analysis of LLMs
  5. 2025A Survey: Towards Privacy and Security in Mobile Large Language Models
  6. 2025Evolving Security in LLMs: A Study of Jailbreak Attacks and Defenses
  7. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks
  8. 2024Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions