# Measuring Data Leakage in Machine-Learning Models with Fisher Information

arXiv: https://arxiv.org/abs/2102.11673 · HTML: https://mlsanity.com/paper/2102.11673/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 11 of 11)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Understanding Membership Inferences on Well-Generalized Learning Models](https://mlsanity.com/paper/1802.04889/) (2018)
- [Towards Measuring Membership Privacy](https://mlsanity.com/paper/1712.09136/) (2017)
- [Modelling and Quantifying Membership Information Leakage in Machine Learning](https://mlsanity.com/paper/2001.10648/) (2020)
- [Inferential Privacy Guarantees for Differentially Private Mechanisms](https://mlsanity.com/paper/1603.01508/) (2016)
- [Black-box Model Inversion Attribute Inference Attacks on Classification Models](https://mlsanity.com/paper/2012.03404/) (2020)

## Built on by (most cited first, 30 of 33)

- [Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practices](https://mlsanity.com/paper/2203.10716/) (2022)
- [Bounding Training Data Reconstruction in Private (Deep) Learning](https://mlsanity.com/paper/2201.12383/) (2022)
- [Unleashing the Power of Randomization in Auditing Differentially Private ML](https://mlsanity.com/paper/2305.18447/) (2023)
- [Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano](https://mlsanity.com/paper/2210.13662/) (2022)
- [Inferring Sensitive Attributes from Model Explanations](https://mlsanity.com/paper/2208.09967/) (2022)
- [Bounding the Invertibility of Privacy-preserving Instance Encoding using Fisher Information](https://mlsanity.com/paper/2305.04146/) (2023)
- [SHAPr: An Efficient and Versatile Membership Privacy Risk Metric for Machine Learning](https://mlsanity.com/paper/2112.02230/) (2021)
- [Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design](https://mlsanity.com/paper/2211.03942/) (2022)
- [Understanding Deep Gradient Leakage via Inversion Influence Functions](https://mlsanity.com/paper/2309.13016/) (2023)
- [Fundamental Limits of Membership Inference Attacks on Machine Learning Models](https://mlsanity.com/paper/2310.13786/) (2023)
- [Practical Bayes-Optimal Membership Inference Attacks](https://mlsanity.com/paper/2505.24089/) (2025)
- [Defending Against Data Reconstruction Attacks in Federated Learning: An Information Theory Approach](https://mlsanity.com/paper/2403.01268/) (2024)
- [Fisher Mask Nodes for Language Model Merging](https://mlsanity.com/paper/2403.09891/) (2024)
- [Measuring and Controlling Split Layer Privacy Leakage Using Fisher Information](https://mlsanity.com/paper/2209.10119/) (2022)
- [Automating Wood Species Detection and Classification in Microscopic Images of Fibrous Materials with Deep Learning](https://mlsanity.com/paper/2307.09588/) (2023)
- [Guarantees of confidentiality via Hammersley-Chapman-Robbins bounds](https://mlsanity.com/paper/2404.02866/) (2024)
- [From Risk to Resilience: Towards Assessing and Mitigating the Risk of Data Reconstruction Attacks in Federated Learning](https://mlsanity.com/paper/2512.15460/) (2025)
- [How Do Input Attributes Impact the Privacy Loss in Differential Privacy?](https://mlsanity.com/paper/2211.10173/) (2022)
- [Uncovering Model Processing Strategies with Non-Negative Per-Example Fisher Factorization](https://mlsanity.com/paper/2310.04649/) (2023)
- [Privacy Amplification for the Gaussian Mechanism via Bounded Support](https://mlsanity.com/paper/2403.05598/) (2024)
- [Federated Causal Discovery Across Heterogeneous Datasets under Latent Confounding](https://mlsanity.com/paper/2603.05149/) (2026)
- [SoK: Memorisation in machine learning](https://mlsanity.com/paper/2311.03075/) (2023)
- [AdaBet: Gradient-free Layer Selection for Efficient Training of Deep Neural Networks](https://mlsanity.com/paper/2510.03101/) (2025)
- [Protecting Data from all Parties: Combining FHE and DP in Federated Learning](https://mlsanity.com/paper/2205.04330/) (2022)
- [SimClone: Detecting Tabular Data Clones using Value Similarity](https://mlsanity.com/paper/2407.12802/) (2024)
- [Bayes-Nash Generative Privacy Against Membership Inference Attacks](https://mlsanity.com/paper/2410.07414/) (2024)
- [Training Set Reconstruction from Differentially Private Forests: How Effective is DP?](https://mlsanity.com/paper/2502.05307/) (2025)
- [Nosy Layers, Noisy Fixes: Tackling DRAs in Federated Learning Systems using Explainable AI](https://mlsanity.com/paper/2505.10942/) (2025)
- [Reducing information dependency does not cause training data privacy. Adversarially non-robust features do](https://mlsanity.com/paper/2607.12354/) (2026)
- [Certified but Private: Scalable Zero-Knowledge Proofs for Neural Network Guarantees](https://mlsanity.com/paper/2608.17070/) (2026)

## Abstract (arXiv)

Machine-learning models contain information about the data they were trained on. This information leaks either through the model itself or through predictions made by the model. Consequently, when the training data contains sensitive attributes, assessing the amount of information leakage is paramount. We propose a method to quantify this leakage using the Fisher information of the model about the data. Unlike the worst-case a priori guarantees of differential privacy, Fisher information loss measures leakage with respect to specific examples, attributes, or sub-populations within the dataset. We motivate Fisher information loss through the Cram'er-Rao bound and delineate the implied threat model. We provide efficient methods to compute Fisher information loss for output-perturbed generalized linear models. Finally, we empirically validate Fisher information loss as a useful measure of information leakage.