mlsanityData to 7 Oct 2026

Last One Standing: A Comparative Analysis of Security and Privacy of Soft Prompt Tuning, LoRA, and In-Context Learning

arXiv 2310.11397 ↗First posted 17 Oct 2023cs.CR cs.LGcited by 13

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Large Language Models (LLMs) are powerful tools for natural language processing, enabling novel applications and user experiences. However, to achieve optimal performance, LLMs often require adaptation with private data, which poses privacy and security challenges. Several techniques have been proposed to adapt LLMs with private data, such as Low-Rank Adaptation (LoRA), Soft Prompt Tuning (SPT), and In-Context Learning (ICL), but their comparative privacy and security properties have not been systematically investigated. In this work, we fill this gap by evaluating the robustness of LoRA, SPT, and ICL against three types of well-established attacks: membership inference, which exposes data leakage (privacy); backdoor, which injects malicious behavior (security); and model stealing, which can violate intellectual property (privacy and security). Our results show that there is no silver bullet for privacy and security in LLM adaptation and each technique has different strengths and weaknesses.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 23 of its 32 references that are in the map. First posted on arXiv 17 Oct 2023.

Builds on

23 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar
  2. 2021Membership Inference Attacks From First Principles7 Dec
  3. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models4 Feb

Other references · 20

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  10. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  11. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  12. 2019High Accuracy and High Fidelity Extraction of Neural Networkscited by 208
8 more
  1. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  2. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  3. 2020Data-Free Model Extractioncited by 121
  4. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 94
  5. 2023Backdoor Attacks for In-Context Learning with Language Modelscited by 92
  6. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 80
  7. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Modelscited by 76
  8. 2019Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?cited by 68

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 5; 2026: 1.

13 papers.

Sort:

By area:

  1. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 106
  5. 2024Membership Inference Attacks Against In-Context Learningcited by 38
  6. 2024Hey, That's My Model! Introducing Chain & Hash, An LLM Fingerprinting Techniquecited by 34
  7. 2023Maatphor: Automated Variant Analysis for Prompt Injection Attackscited by 12
  8. 2025LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Modelscited by 7
  9. 2024Understanding Data Importance in Machine Learning Attacks: Does Valuable Data Pose Greater Harm?cited by 5
  10. 2025SoK: Data Reconstruction Attacks Against Machine Learning Models: Definition, Metrics, and Benchmarkcited by 5
  11. 2024AIDBench: A benchmark for evaluating the authorship identification capability of large language modelscited by 4
  12. 2025Membership Inference Attacks on In-Context Examples in LLM-based Recommender Systemscited by 0
1 more
  1. 2026Context Inference Attacks Without Jailbreakscited by 0