# Improved Membership Inference Attacks Against Language Classification Models

arXiv: https://arxiv.org/abs/2310.07219 · HTML: https://mlsanity.com/paper/2310.07219/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 11 Oct 2023.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 14 of 14)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [Information Leakage in Embedding Models](https://mlsanity.com/paper/2004.00053/) (2020)
- [Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks](https://mlsanity.com/paper/2203.03929/) (2022)
- [ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning](https://mlsanity.com/paper/2007.09339/) (2020)
- [Membership Inference Attack Susceptibility of Clinical Language Models](https://mlsanity.com/paper/2104.08305/) (2021)
- [Membership Inference on Word Embedding and Beyond](https://mlsanity.com/paper/2106.11384/) (2021)
- [Ensemble-based Blackbox Attacks on Dense Prediction](https://mlsanity.com/paper/2303.14304/) (2023)

## Built on by (most cited first, 7 of 7)

- [Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models](https://mlsanity.com/paper/2310.15007/) (2023)
- [Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generation](https://mlsanity.com/paper/2405.20446/) (2024)
- [Undesirable Memorization in Large Language Models: A Survey](https://mlsanity.com/paper/2410.02650/) (2024)
- [SoK: Reducing the Vulnerability of Fine-tuned Language Models to Membership Inference Attacks](https://mlsanity.com/paper/2403.08481/) (2024)
- [Membership Inference Attacks Against Time-Series Models](https://mlsanity.com/paper/2407.02870/) (2024)
- [Privacy Risks in Time Series Forecasting: User- and Record-Level Membership Inference](https://mlsanity.com/paper/2509.04169/) (2025)
- [ReproMIA: A Comprehensive Analysis of Model Reprogramming for Proactive Membership Inference Attacks](https://mlsanity.com/paper/2603.28942/) (2026)

## Abstract (arXiv)

Artificial intelligence systems are prevalent in everyday life, with use cases in retail, manufacturing, health, and many other fields. With the rise in AI adoption, associated risks have been identified, including privacy risks to the people whose data was used to train models. Assessing the privacy risks of machine learning models is crucial to enabling knowledgeable decisions on whether to use, deploy, or share a model. A common approach to privacy risk assessment is to run one or more known attacks against the model and measure their success rate. We present a novel framework for running membership inference attacks against classification models. Our framework takes advantage of the ensemble method, generating many specialized attack models for different subsets of the data. We show that this approach achieves higher accuracy than either a single attack model or an attack model per class label, both on classical and language classification tasks.