# Noisy Neighbors: Efficient membership inference attacks against LLMs

arXiv: https://arxiv.org/abs/2406.16565 · HTML: https://mlsanity.com/paper/2406.16565/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [On the Importance of Difficulty Calibration in Membership Inference Attacks](https://mlsanity.com/paper/2111.08440/) (2021)
- [MetaPoison: Practical General-purpose Clean-label Data Poisoning](https://mlsanity.com/paper/2004.00225/) (2020)
- [On the Exploitability of Instruction Tuning](https://mlsanity.com/paper/2306.17194/) (2023)
- [Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models](https://mlsanity.com/paper/2305.14710/) (2023)
- [Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets](https://mlsanity.com/paper/2204.00032/) (2022)
- [Property Inference From Poisoning](https://mlsanity.com/paper/2101.11073/) (2021)
- [Nationality Bias in Text Generation](https://mlsanity.com/paper/2302.02463/) (2023)

## Built on by (most cited first, 21 of 21)

- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Membership Inference Attacks on Large-Scale Models: A Survey](https://mlsanity.com/paper/2503.19338/) (2025)
- [Recent Advances in Attack and Defense Approaches of Large Language Models](https://mlsanity.com/paper/2409.03274/) (2024)
- [Nob-MIAs: Non-biased Membership Inference Attacks Assessment on Large Language Models with Ex-Post Dataset Construction](https://mlsanity.com/paper/2408.05968/) (2024)
- [Hey, That's My Data! Token-Only Dataset Inference in Large Language Models](https://mlsanity.com/paper/2506.06057/) (2025)
- [Smaller = Weaker? Benchmarking Robustness of Quantized LLMs in Code Generation](https://mlsanity.com/paper/2506.22776/) (2025)
- [SoK: Semantic Privacy in Large Language Models](https://mlsanity.com/paper/2506.23603/) (2025)
- [When Tables Leak: Attacking String Memorization in LLM-Based Tabular Data Generation](https://mlsanity.com/paper/2512.08875/) (2025)
- [How Privacy-Savvy Are Large Language Models? A Case Study on Compliance and Privacy Technical Review](https://mlsanity.com/paper/2409.02375/) (2024)
- [What Really is a Member? Discrediting Membership Inference via Poisoning](https://mlsanity.com/paper/2506.06003/) (2025)
- [Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessment](https://mlsanity.com/paper/2508.08629/) (2025)
- [ISACL: Internal State Analyzer for Copyrighted Training Data Leakage](https://mlsanity.com/paper/2508.17767/) (2025)
- [Membership Inference Attacks against Large Audio Language Models](https://mlsanity.com/paper/2603.28378/) (2026)
- [Data Compressibility Quantifies LLM Memorization](https://mlsanity.com/paper/2507.06056/) (2025)
- [A Survey: Towards Privacy and Security in Mobile Large Language Models](https://mlsanity.com/paper/2509.02411/) (2025)
- [LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems](https://mlsanity.com/paper/2509.10682/) (2025)
- [Exploring Membership Inference Vulnerabilities in Clinical Large Language Models](https://mlsanity.com/paper/2510.18674/) (2025)
- [REMIND: Input Loss Landscapes Reveal Residual Memorization in Post-Unlearning LLMs](https://mlsanity.com/paper/2511.04228/) (2025)
- [Sensitivity-Aware Retrieval-Augmented Intent Clarification](https://mlsanity.com/paper/2603.06025/) (2026)
- [Reconstruction of Personally Identifiable Information from Proprietary Data in Supervised Fine-Tuned Models](https://mlsanity.com/paper/2605.12264/) (2026)
- [Calibrating One-Round Membership Inference with Neighbors](https://mlsanity.com/paper/2609.36331/) (2026)

## Abstract (arXiv)

The potential of transformer-based LLMs risks being hindered by privacy concerns due to their reliance on extensive datasets, possibly including sensitive information. Regulatory measures like GDPR and CCPA call for using robust auditing tools to address potential privacy issues, with Membership Inference Attacks (MIA) being the primary method for assessing LLMs' privacy risks. Differently from traditional MIA approaches, often requiring computationally intensive training of additional models, this paper introduces an efficient methodology that generates textitnoisy neighbors for a target sample by adding stochastic noise in the embedding space, requiring operating the target model in inference mode only. Our findings demonstrate that this approach closely matches the effectiveness of employing shadow models, showing its usability in practical privacy auditing scenarios.