# Students Parrot Their Teachers: Membership Inference on Model Distillation

arXiv: https://arxiv.org/abs/2303.03446 · HTML: https://mlsanity.com/paper/2303.03446/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)
- Knowledge distillation / [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/)

## Builds on (most cited first, 28 of 28)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Patient Knowledge Distillation for BERT Model Compression](https://mlsanity.com/paper/1908.09355/) (2019)
- [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Paraphrasing Complex Network: Network Compression via Factor Transfer](https://mlsanity.com/paper/1802.04977/) (2018)
- [High Accuracy and High Fidelity Extraction of Neural Networks](https://mlsanity.com/paper/1909.01838/) (2019)
- [Membership Leakage in Label-Only Exposures](https://mlsanity.com/paper/2007.15528/) (2020)
- [On the Importance of Difficulty Calibration in Membership Inference Attacks](https://mlsanity.com/paper/2111.08440/) (2021)
- [The Privacy Onion Effect: Memorization is Relative](https://mlsanity.com/paper/2206.10469/) (2022)
- [Revisiting Membership Inference Under Realistic Assumptions](https://mlsanity.com/paper/2005.10881/) (2020)
- [Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets](https://mlsanity.com/paper/2204.00032/) (2022)
- [Less is More: Task-aware Layer-wise Distillation for Language Model Compression](https://mlsanity.com/paper/2210.01351/) (2022)
- [Understanding Membership Inferences on Well-Generalized Learning Models](https://mlsanity.com/paper/1802.04889/) (2018)
- [Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture](https://mlsanity.com/paper/2110.08324/) (2021)
- [Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries](https://mlsanity.com/paper/2210.10750/) (2022)
- [Amplifying Membership Exposure via Data Poisoning](https://mlsanity.com/paper/2211.00463/) (2022)
- [Differentially Private Model Compression](https://mlsanity.com/paper/2206.01838/) (2022)

## Built on by (most cited first, 30 of 33)

- [Red Teaming Language Model Detectors with Language Models](https://mlsanity.com/paper/2305.19713/) (2023)
- [User Inference Attacks on Large Language Models](https://mlsanity.com/paper/2310.09266/) (2023)
- [Teach LLMs to Phish: Stealing Private Information from Language Models](https://mlsanity.com/paper/2403.00871/) (2024)
- [Auditing f-Differential Privacy in One Run](https://mlsanity.com/paper/2410.22235/) (2024)
- [Privacy Auditing of Large Language Models](https://mlsanity.com/paper/2503.06808/) (2025)
- [Language Models May Verbatim Complete Text They Were Not Explicitly Trained On](https://mlsanity.com/paper/2503.17514/) (2025)
- [TMI! Finetuned Models Leak Private Information from their Pretraining Data](https://mlsanity.com/paper/2306.01181/) (2023)
- [Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training](https://mlsanity.com/paper/2502.15680/) (2025)
- [TUNI: A Textual Unimodal Detector for Identity Inference in CLIP Models](https://mlsanity.com/paper/2405.14517/) (2024)
- [GLiRA: Black-Box Membership Inference Attack via Knowledge Distillation](https://mlsanity.com/paper/2405.07562/) (2024)
- [Dataset Size Recovery from LoRA Weights](https://mlsanity.com/paper/2406.19395/) (2024)
- [Generalizing Trust: Weak-to-Strong Trustworthiness in Language Models](https://mlsanity.com/paper/2501.00418/) (2024)
- [Cascading and Proxy Membership Inference Attacks](https://mlsanity.com/paper/2507.21412/) (2025)
- [Antidistillation Fingerprinting](https://mlsanity.com/paper/2602.03812/) (2026)
- [DPDR: Gradient Decomposition and Reconstruction for Differentially Private Deep Learning](https://mlsanity.com/paper/2406.02744/) (2024)
- [A hierarchical approach for assessing the vulnerability of tree-based classification models to membership inference attack](https://mlsanity.com/paper/2502.09396/) (2025)
- [Membership and Memorization in LLM Knowledge Distillation](https://mlsanity.com/paper/2508.07054/) (2025)
- [Imitative Membership Inference Attack](https://mlsanity.com/paper/2509.06796/) (2025)
- [Learning-Based Difficulty Calibration for Enhanced Membership Inference Attacks](https://mlsanity.com/paper/2401.04929/) (2024)
- [Memorization Inheritance in Sequence-Level Knowledge Distillation for Neural Machine Translation](https://mlsanity.com/paper/2502.01491/) (2025)
- [Quantifying the Effect of Test Set Contamination on Generative Evaluations](https://mlsanity.com/paper/2601.04301/) (2026)
- [Memorization Dynamics in Knowledge Distillation for Language Models](https://mlsanity.com/paper/2601.15394/) (2026)
- [Scaling Model-Generated Distillation Data Can Make Latent Teacher Traits More Recoverable](https://mlsanity.com/paper/2608.26958/) (2026)
- [Membership Inference Attacks on DNNs using Adversarial Perturbations](https://mlsanity.com/paper/2307.05193/) (2023)
- [On Membership Inference Attacks in Knowledge Distillation](https://mlsanity.com/paper/2505.11837/) (2025)
- [SoK: Challenges in Tabular Membership Inference Attacks](https://mlsanity.com/paper/2601.15874/) (2026)
- [Knowledge Distillation Must Account for What It Loses](https://mlsanity.com/paper/2604.25110/) (2026)
- [FictionalQA: A Dataset for Studying Memorization and Knowledge Acquisition](https://mlsanity.com/paper/2506.05639/) (2025)
- [Synthetic Adaptive Guided Embeddings (SAGE): A Novel Knowledge Distillation Method](https://mlsanity.com/paper/2508.14783/) (2025)
- [Curation Leaks: Membership Inference Attacks against Data Curation for Machine Learning](https://mlsanity.com/paper/2603.00811/) (2026)

## Abstract (arXiv)

Model distillation is frequently proposed as a technique to reduce the privacy leakage of machine learning. These empirical privacy defenses rely on the intuition that distilled ``student'' models protect the privacy of training data, as they only interact with this data indirectly through a ``teacher'' model. In this work, we design membership inference attacks to systematically study the privacy provided by knowledge distillation to both the teacher and student training sets. Our new attacks show that distillation alone provides only limited privacy across a number of domains. We explain the success of our attacks on distillation by showing that membership inference attacks on a private dataset can succeed even if the target model is *never* queried on any actual training points, but only on inputs whose predictions are highly influenced by training data. Finally, we show that our attacks are strongest when student and teacher sets are similar, or when the attacker can poison the teacher set.