mlsanityData to 14 Oct 2026

Win-k: Improved Membership Inference Attacks on Small Language Models

arXiv 2508.01268 ↗First posted 2 Aug 2025cs.AI cs.CRcited by 1

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Small language models (SLMs) are increasingly valued for their efficiency and deployability in resource-constrained environments, making them useful for on-device, privacy-sensitive, and edge computing applications. On the other hand, membership inference attacks (MIAs), which aim to determine whether a given sample was used in a model's training, are an important threat with serious privacy and intellectual property implications. In this paper, we study MIAs on SLMs. Although MIAs were shown to be effective on large language models (LLMs), they are relatively less studied on emerging SLMs, and furthermore, their effectiveness decreases as models get smaller. Motivated by this finding, we propose a new MIA called win-k, which builds on top of a state-of-the-art attack (min-k). We experimentally evaluate win-k by comparing it with five existing MIAs using three datasets and eight SLMs. Results show that win-k outperforms existing MIAs in terms of AUROC, TPR @ 1% FPR, and FPR @ 99% TPR metrics, especially on smaller models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 24 of its 30 references that are in the map. First posted on arXiv 2 Aug 2025.

Builds on

24 of 30 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Nob-MIAs: Non-biased Membership Inference Attacks Assessment on Large Language Models with Ex-Post Dataset Construction12 Aug
  2. 2024LLM Dataset Inference: Did you train on my dataset?10 Jun
  3. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  4. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models23 Oct
  5. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparison29 May
  6. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks8 Mar

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  5. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  7. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  8. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  9. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  10. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  11. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 129
  12. 2024Small Language Models: Survey, Measurements, and Insightscited by 112
6 more
  1. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 103
  2. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  3. 2023User Inference Attacks on Large Language Modelscited by 36
  4. 2024A Survey of Small Language Modelscited by 31
  5. 2023FwdLLM: Efficient FedLLM using Forward Gradientcited by 26
  6. 2024Fine-tuning Smaller Language Models for Question Answering over Financial Documentscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026VLALeaks: Membership Inference Attacks against Vision-Language-Action Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models
  2. 2025Membership Inference Attacks on Large-Scale Models: A Survey
  3. 2025Exploring the limits of strong membership inference attacks on large language models
  4. 2024Order of Magnitude Speedups for LLM Membership Inference
  5. 2025Fast-MIA: Efficient and Scalable Membership Inference for LLMs
  6. 2025On the Effectiveness of Membership Inference in Targeted Data Extraction from Large Language Models
  7. 2024MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector
  8. 2024Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models