# Do Parameters Reveal More than Loss for Membership Inference?

arXiv: https://arxiv.org/abs/2406.11544 · HTML: https://mlsanity.com/paper/2406.11544/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 31 of its 43 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 31)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Stochastic Gradient Descent as Approximate Bayesian Inference](https://mlsanity.com/paper/1704.04289/) (2017)
- [White-box vs Black-box: Bayes Optimal Strategies for Membership Inference](https://mlsanity.com/paper/1908.11229/) (2019)
- [Enhanced Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/2111.09679/) (2021)
- [Emergent and Predictable Memorization in Large Language Models](https://mlsanity.com/paper/2304.11158/) (2023)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning](https://mlsanity.com/paper/1812.00910/) (2018)
- [ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning](https://mlsanity.com/paper/2007.09339/) (2020)
- [Scalable Membership Inference Attacks via Quantile Regression](https://mlsanity.com/paper/2307.03694/) (2023)
- [SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning](https://mlsanity.com/paper/2212.10986/) (2022)
- [Membership Inference Attacks and Defenses in Neural Network Pruning](https://mlsanity.com/paper/2202.03335/) (2022)
- [Strength of Minibatch Noise in SGD](https://mlsanity.com/paper/2102.05375/) (2021)
- [Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent](https://mlsanity.com/paper/2012.03636/) (2020)
- [Power-law escape rate of SGD](https://mlsanity.com/paper/2105.09557/) (2021)
- [Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deployment](https://mlsanity.com/paper/2304.06929/) (2023)
- [MoPe: Model Perturbation-based Privacy Attacks on Language Models](https://mlsanity.com/paper/2310.14369/) (2023)
- [Membership Inference Attack Using Self Influence Functions](https://mlsanity.com/paper/2205.13680/) (2022)
- [Leave-one-out Distinguishability in Machine Learning](https://mlsanity.com/paper/2309.17310/) (2023)
- [PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining](https://mlsanity.com/paper/2402.09477/) (2024)
- [Parameters or Privacy: A Provable Tradeoff Between Overparameterization and Membership Inference](https://mlsanity.com/paper/2202.01243/) (2022)
- [Is My Data in Your AI? Membership Inference Test (MINT) applied to Face Biometrics](https://mlsanity.com/paper/2402.09225/) (2024)
- [Investigating the Effect of Misalignment on Membership Privacy in the White-box Setting](https://mlsanity.com/paper/2306.05093/) (2023)
- [Noise Balance and Stationary Distribution of Stochastic Gradient Descent](https://mlsanity.com/paper/2308.06671/) (2023)
- [Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning](https://mlsanity.com/paper/2310.03838/) (2023)
- [Bayesian interpretation of SGD as Ito process](https://mlsanity.com/paper/1911.09011/) (2019)
- [Some Targets Are Harder to Identify than Others: Quantifying the Target-dependent Membership Leakage](https://mlsanity.com/paper/2402.10065/) (2024)

## Built on by (most cited first, 10 of 10)

- [(Token-Level) InfoRMIA: Stronger Membership Inference and Memorization Assessment for LLMs](https://mlsanity.com/paper/2510.05582/) (2025)
- [ImpMIA: Leveraging Implicit Bias for Membership Inference Attack](https://mlsanity.com/paper/2510.10625/) (2025)
- [SoK: Challenges in Tabular Membership Inference Attacks](https://mlsanity.com/paper/2601.15874/) (2026)
- [AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective](https://mlsanity.com/paper/2603.24857/) (2026)
- [How Learning Dynamics Drive Adversarially Robust Generalization?](https://mlsanity.com/paper/2410.07719/) (2024)
- [Empirical Comparison of Membership Inference Attacks in Deep Transfer Learning](https://mlsanity.com/paper/2510.05753/) (2025)
- [Assessing Per-Sample Membership Inference Vulnerability without Retraining](https://mlsanity.com/paper/2602.15919/) (2026)
- [A Critical Review on the Effectiveness and Privacy Threats of Membership Inference Attacks](https://mlsanity.com/paper/2603.22987/) (2026)
- [Predicting Privacy Leakage from Weight Spectral Density](https://mlsanity.com/paper/2609.11780/) (2026)
- [SoK: Rethinking Membership Inference in Tabular Machine Learning](https://mlsanity.com/paper/2610.18486/) (2026)

## Abstract (arXiv)

Membership inference attacks are used as a key tool for disclosure auditing. They aim to infer whether an individual record was used to train a model. While such evaluations are useful to demonstrate risk, they are computationally expensive and often make strong assumptions about potential adversaries' access to models and training environments, and thus do not provide tight bounds on leakage from potential attacks. We show how prior claims around black-box access being sufficient for optimal membership inference do not hold for stochastic gradient descent, and that optimal membership inference indeed requires white-box access. Our theoretical results lead to a new white-box inference attack, IHA (Inverse Hessian Attack), that explicitly uses model parameters by taking advantage of computing inverse-Hessian vector products. Our results show that both auditors and adversaries may be able to benefit from access to model parameters, and we advocate for further research into white-box methods for membership inference.