# Enhancing Privacy against Inversion Attacks in Federated Learning by using Mixing Gradients Strategies

arXiv: https://arxiv.org/abs/2204.12495 · HTML: https://mlsanity.com/paper/2204.12495/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 20 of 20)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Differentially Private Learning Needs Better Features (or Much More Data)](https://mlsanity.com/paper/2011.11660/) (2020)
- [Stealing Hyperparameters in Machine Learning](https://mlsanity.com/paper/1802.05351/) (2018)
- [R-GAP: Recursive Gradient Attack on Privacy](https://mlsanity.com/paper/2010.07733/) (2020)
- [InstaHide: Instance-hiding Schemes for Private Distributed Learning](https://mlsanity.com/paper/2010.02772/) (2020)
- [A Framework for Evaluating Gradient Leakage Attacks in Federated Learning](https://mlsanity.com/paper/2004.10397/) (2020)
- [Is Private Learning Possible with Instance Encoding?](https://mlsanity.com/paper/2011.05315/) (2020)
- [Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables](https://mlsanity.com/paper/1703.00091/) (2017)
- [Minimal Model Structure Analysis for Input Reconstruction in Federated Learning](https://mlsanity.com/paper/2010.15718/) (2020)

## Built on by (most cited first, 5 of 5)

- [Privacy-preserving quantum federated learning via gradient hiding](https://mlsanity.com/paper/2312.04447/) (2023)
- [Expressive variational quantum circuits provide inherent privacy in federated learning](https://mlsanity.com/paper/2309.13002/) (2023)
- [Prospects of Privacy Advantage in Quantum Machine Learning](https://mlsanity.com/paper/2405.08801/) (2024)
- [Adversarial Robustness in Distributed Quantum Machine Learning](https://mlsanity.com/paper/2508.11848/) (2025)
- [PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMs](https://mlsanity.com/paper/2506.02965/) (2025)

## Abstract (arXiv)

Federated learning reduces the risk of information leakage, but remains vulnerable to attacks. We investigate how several neural network design decisions can defend against gradients inversion attacks. We show that overlapping gradients provides numerical resistance to gradient inversion on the highly vulnerable dense layer. Specifically, we propose to leverage batching to maximise mixing of gradients by choosing an appropriate loss function and drawing identical labels. We show that otherwise it is possible to directly recover all vectors in a mini-batch without any numerical optimisation due to the de-mixing nature of the cross entropy loss. To accurately assess data recovery, we introduce an absolute variation distance (AVD) metric for information leakage in images, derived from total variation. In contrast to standard metrics, e.g. Mean Squared Error or Structural Similarity Index, AVD offers a continuous metric for extracting information in noisy images. Finally, our empirical results on information recovery from various inversion attacks and training performance supports our defense strategies. These strategies are also shown to be useful for deep convolutional neural networks such as LeNET for image recognition. We hope that this study will help guide the development of further strategies that achieve a trustful federation policy.