# A Quantitative Metric for Privacy Leakage in Federated Learning

arXiv: https://arxiv.org/abs/2102.13472 · HTML: https://mlsanity.com/paper/2102.13472/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [A Survey on Homomorphic Encryption Schemes: Theory and Implementation](https://mlsanity.com/paper/1704.03578/) (2017)
- [Federated Visual Classification with Real-World Data Distribution](https://mlsanity.com/paper/2003.08082/) (2020)
- [Scalable Mutual Information Estimation using Dependence Graphs](https://mlsanity.com/paper/1801.09125/) (2018)
- [On-line Adaptative Curriculum Learning for GANs](https://mlsanity.com/paper/1808.00020/) (2018)
- [Mutual Information Gradient Estimation for Representation Learning](https://mlsanity.com/paper/2005.01123/) (2020)
- [FedSmart: An Auto Updating Federated Learning Optimization Mechanism](https://mlsanity.com/paper/2009.07455/) (2020)

## Built on by (most cited first, 12 of 12)

- [A Survey on Gradient Inversion: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2206.07284/) (2022)
- [Gradient Obfuscation Gives a False Sense of Security in Federated Learning](https://mlsanity.com/paper/2206.04055/) (2022)
- [Subject Membership Inference Attacks in Federated Learning](https://mlsanity.com/paper/2206.03317/) (2022)
- [Gradient Leakage Attack Resilient Deep Learning](https://mlsanity.com/paper/2112.13178/) (2021)
- [Federated Learning with Dynamic Transformer for Text to Speech](https://mlsanity.com/paper/2107.08795/) (2021)
- [FederatedTrust: A Solution for Trustworthy Federated Learning](https://mlsanity.com/paper/2302.09844/) (2023)
- [Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimization](https://mlsanity.com/paper/2407.09324/) (2024)
- [Cali3F: Calibrated Fast Fair Federated Recommendation System](https://mlsanity.com/paper/2205.13121/) (2022)
- [Enhancing Privacy in Face Analytics Using Fully Homomorphic Encryption](https://mlsanity.com/paper/2404.16255/) (2024)
- [Prototype-Based Layered Federated Cross-Modal Hashing](https://mlsanity.com/paper/2210.15678/) (2022)
- [Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarization](https://mlsanity.com/paper/2304.11524/) (2023)
- [Privacy-Preserving Collaborative Split Learning Framework for Smart Grid Load Forecasting](https://mlsanity.com/paper/2403.01438/) (2024)

## Abstract (arXiv)

In the federated learning system, parameter gradients are shared among participants and the central modulator, while the original data never leave their protected source domain. However, the gradient itself might carry enough information for precise inference of the original data. By reporting their parameter gradients to the central server, client datasets are exposed to inference attacks from adversaries. In this paper, we propose a quantitative metric based on mutual information for clients to evaluate the potential risk of information leakage in their gradients. Mutual information has received increasing attention in the machine learning and data mining community over the past few years. However, existing mutual information estimation methods cannot handle high-dimensional variables. In this paper, we propose a novel method to approximate the mutual information between the high-dimensional gradients and batched input data. Experimental results show that the proposed metric reliably reflect the extent of information leakage in federated learning. In addition, using the proposed metric, we investigate the influential factors of risk level. It is proven that, the risk of information leakage is related to the status of the task model, as well as the inherent data distribution.