mlsanityData to 7 Oct 2026

Noisy Neighbors: Efficient membership inference attacks against LLMs

arXiv 2406.16565 ↗First posted 24 Jun 2024cs.CR cs.LGcited by 20

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

The potential of transformer-based LLMs risks being hindered by privacy concerns due to their reliance on extensive datasets, possibly including sensitive information. Regulatory measures like GDPR and CCPA call for using robust auditing tools to address potential privacy issues, with Membership Inference Attacks (MIA) being the primary method for assessing LLMs' privacy risks. Differently from traditional MIA approaches, often requiring computationally intensive training of additional models, this paper introduces an efficient methodology that generates textitnoisy neighbors for a target sample by adding stochastic noise in the embedding space, requiring operating the target model in inference mode only. Our findings demonstrate that this approach closely matches the effectiveness of employing shadow models, showing its usability in practical privacy auditing scenarios.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 17 of its 27 references that are in the map. First posted on arXiv 24 Jun 2024.

Builds on

17 of 27 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  2. 2021Membership Inference Attacks From First Principles7 Dec
  3. 2021On the Importance of Difficulty Calibration in Membership Inference Attacks15 Nov

Other references · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Pointer Sentinel Mixture Modelscited by 2,992
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2023Extracting Training Data from Diffusion Modelscited by 723
  8. 2013The Composition Theorem for Differential Privacycited by 290
  9. 2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 114
  10. 2023On the Exploitability of Instruction Tuningcited by 110
  11. 2023Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Modelscited by 100
  12. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 94
2 more
  1. 2021Property Inference From Poisoningcited by 56
  2. 2023Nationality Bias in Text Generationcited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 14; 2026: 4.

20 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2025Membership Inference Attacks on Large-Scale Models: A Surveycited by 17
  3. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
  4. 2025Hey, That's My Data! Token-Only Dataset Inference in Large Language Modelscited by 3
  5. 2025Smaller = Weaker? Benchmarking Robustness of Quantized LLMs in Code Generationcited by 3
  6. 2025SoK: Semantic Privacy in Large Language Modelscited by 3
  7. 2025When Tables Leak: Attacking String Memorization in LLM-Based Tabular Data Generationcited by 3
  8. 2024How Privacy-Savvy Are Large Language Models? A Case Study on Compliance and Privacy Technical Reviewcited by 2
  9. 2025What Really is a Member? Discrediting Membership Inference via Poisoningcited by 2
  10. 2025Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessmentcited by 2
  11. 2025ISACL: Internal State Analyzer for Copyrighted Training Data Leakagecited by 2
  12. 2025Data Compressibility Quantifies LLM Memorizationcited by 1
8 more
  1. 2025A Survey: Towards Privacy and Security in Mobile Large Language Modelscited by 1
  2. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  3. 2025Exploring Membership Inference Vulnerabilities in Clinical Large Language Modelscited by 1
  4. 2026Membership Inference Attacks against Large Audio Language Modelscited by 1
  5. 2025REMIND: Input Loss Landscapes Reveal Residual Memorization in Post-Unlearning LLMscited by 0
  6. 2026Sensitivity-Aware Retrieval-Augmented Intent Clarificationcited by 0
  7. 2026Reconstruction of Personally Identifiable Information from Proprietary Data in Supervised Fine-Tuned Modelscited by 0
  8. 2026Calibrating One-Round Membership Inference with Neighborscited by 0