# Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries

arXiv: https://arxiv.org/abs/2210.10750 · HTML: https://mlsanity.com/paper/2210.10750/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Systematic Evaluation of Privacy Risks of Machine Learning Models](https://mlsanity.com/paper/2003.10595/) (2020)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Patches Are All You Need?](https://mlsanity.com/paper/2201.09792/) (2022)
- [Datamodels: Predicting Predictions from Training Data](https://mlsanity.com/paper/2202.00622/) (2022)
- [Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference](https://mlsanity.com/paper/1906.11798/) (2019)
- [The Privacy Onion Effect: Memorization is Relative](https://mlsanity.com/paper/2206.10469/) (2022)
- [Measuring Forgetting of Memorized Training Examples](https://mlsanity.com/paper/2207.00099/) (2022)
- [Membership Inference Attacks by Exploiting Loss Trajectory](https://mlsanity.com/paper/2208.14933/) (2022)
- [Analyzing Information Leakage of Updates to Natural Language Models](https://mlsanity.com/paper/1912.07942/) (2019)
- [Eluding Secure Aggregation in Federated Learning via Model Inconsistency](https://mlsanity.com/paper/2111.07380/) (2021)
- [Stochastic Training is Not Necessary for Generalization](https://mlsanity.com/paper/2109.14119/) (2021)
- [Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspective](https://mlsanity.com/paper/2203.08124/) (2022)
- [Algorithms that Remember: Model Inversion Attacks and Data Protection Law](https://mlsanity.com/paper/1807.04644/) (2018)
- [Parameters or Privacy: A Provable Tradeoff Between Overparameterization and Membership Inference](https://mlsanity.com/paper/2202.01243/) (2022)
- [Quantifying Membership Inference Vulnerability via Generalization Gap and Other Model Metrics](https://mlsanity.com/paper/2009.05669/) (2020)
- [How to Combine Membership-Inference Attacks on Multiple Updated Models](https://mlsanity.com/paper/2205.06369/) (2022)

## Built on by (most cited first, 30 of 40)

- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Privacy Auditing with One (1) Training Run](https://mlsanity.com/paper/2305.08846/) (2023)
- [Low-Cost High-Power Membership Inference Attacks](https://mlsanity.com/paper/2312.03262/) (2023)
- [Detecting, Explaining, and Mitigating Memorization in Diffusion Models](https://mlsanity.com/paper/2407.21720/) (2024)
- [Scalable Membership Inference Attacks via Quantile Regression](https://mlsanity.com/paper/2307.03694/) (2023)
- [Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study](https://mlsanity.com/paper/2310.00108/) (2023)
- [Evaluations of Machine Learning Privacy Defenses are Misleading](https://mlsanity.com/paper/2404.17399/) (2024)
- [Students Parrot Their Teachers: Membership Inference on Model Distillation](https://mlsanity.com/paper/2303.03446/) (2023)
- [Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models](https://mlsanity.com/paper/2404.01231/) (2024)
- [Is Difficulty Calibration All We Need? Towards More Practical Membership Inference Attacks](https://mlsanity.com/paper/2409.00426/) (2024)
- [Leave-one-out Distinguishability in Machine Learning](https://mlsanity.com/paper/2309.17310/) (2023)
- [Inf2Guard: An Information-Theoretic Framework for Learning Privacy-Preserving Representations against Inference Attacks](https://mlsanity.com/paper/2403.02116/) (2024)
- [OSLO: One-Shot Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2405.16978/) (2024)
- [MIST: Defending Against Membership Inference Attacks Through Membership-Invariant Subspace Training](https://mlsanity.com/paper/2311.00919/) (2023)
- [The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD](https://mlsanity.com/paper/2410.06186/) (2024)
- [GLiRA: Black-Box Membership Inference Attack via Knowledge Distillation](https://mlsanity.com/paper/2405.07562/) (2024)
- [U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AI](https://mlsanity.com/paper/2406.15386/) (2024)
- [A Method to Facilitate Membership Inference Attacks in Deep Learning Models](https://mlsanity.com/paper/2407.01919/) (2024)
- [Cascading and Proxy Membership Inference Attacks](https://mlsanity.com/paper/2507.21412/) (2025)
- [Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning](https://mlsanity.com/paper/2310.03838/) (2023)
- [The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditing](https://mlsanity.com/paper/2506.16666/) (2025)
- [Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts](https://mlsanity.com/paper/2510.13451/) (2025)
- [Has My System Prompt Been Used? Large Language Model Prompt Membership Inference](https://mlsanity.com/paper/2502.09974/) (2025)
- [SolidMark: Evaluating Image Memorization in Generative Models](https://mlsanity.com/paper/2503.00592/) (2025)
- [Imitative Membership Inference Attack](https://mlsanity.com/paper/2509.06796/) (2025)
- [Reasoning's Razor: Reasoning Improves Accuracy but Can Hurt Recall at Critical Operating Points in Safety and Hallucination Detection](https://mlsanity.com/paper/2510.21049/) (2025)
- [Revisiting Privacy Leakage in Machine Unlearning: Membership Inference Beyond the Forgotten Set](https://mlsanity.com/paper/2605.01129/) (2026)
- [Better Membership Inference Privacy Measurement through Discrepancy](https://mlsanity.com/paper/2405.15140/) (2024)
- [Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence](https://mlsanity.com/paper/2407.14710/) (2024)
- [Apollo: A Posteriori Label-Only Membership Inference Attack Towards Machine Unlearning](https://mlsanity.com/paper/2506.09923/) (2025)

## Abstract (arXiv)

As industrial applications are increasingly automated by machine learning models, enforcing personal data ownership and intellectual property rights requires tracing training data back to their rightful owners. Membership inference algorithms approach this problem by using statistical techniques to discern whether a target sample was included in a model's training set. However, existing methods only utilize the unaltered target sample or simple augmentations of the target to compute statistics. Such a sparse sampling of the model's behavior carries little information, leading to poor inference capabilities. In this work, we use adversarial tools to directly optimize for queries that are discriminative and diverse. Our improvements achieve significantly more accurate membership inference than existing methods, especially in offline scenarios and in the low false-positive regime which is critical in legal settings. Code is available at https://github.com/YuxinWenRick/canary-in-a-coalmine.