# Differential Privacy Meets Federated Learning under Communication Constraints

arXiv: https://arxiv.org/abs/2101.12240 · HTML: https://mlsanity.com/paper/2101.12240/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 28 Jan 2021.

## Builds on (most cited first, 22 of 22)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning](https://mlsanity.com/paper/1705.07878/) (2017)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [SCAFFOLD: Stochastic Controlled Averaging for Federated Learning](https://mlsanity.com/paper/1910.06378/) (2019)
- [cpSGD: Communication-efficient and differentially-private distributed SGD](https://mlsanity.com/paper/1805.10559/) (2018)
- [Revisiting Distributed Synchronous SGD](https://mlsanity.com/paper/1702.05800/) (2017)
- [The Discrete Gaussian for Differential Privacy](https://mlsanity.com/paper/2004.00010/) (2020)
- [Communication Efficient Distributed Optimization using an Approximate Newton-type Method](https://mlsanity.com/paper/1312.7853/) (2013)
- [First Analysis of Local GD on Heterogeneous Data](https://mlsanity.com/paper/1909.04715/) (2019)
- [FedDANE: A Federated Newton-Type Method](https://mlsanity.com/paper/2001.01920/) (2020)
- [LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy](https://mlsanity.com/paper/2007.15789/) (2020)
- [Artificial Intelligence-Enabled Cellular Networks: A Critical Path to Beyond-5G and 6G](https://mlsanity.com/paper/1907.07862/) (2019)
- [Dual Policy Distillation](https://mlsanity.com/paper/2006.04061/) (2020)
- [Secure and Utility-Aware Data Collection with Condensed Local Differential Privacy](https://mlsanity.com/paper/1905.06361/) (2019)

## Built on by (most cited first, 16 of 16)

- [Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models](https://mlsanity.com/paper/2305.11414/) (2023)
- [Layered Randomized Quantization for Communication-Efficient and Privacy-Preserving Distributed Learning](https://mlsanity.com/paper/2312.07060/) (2023)
- [Differentially Private Vertical Federated Learning](https://mlsanity.com/paper/2211.06782/) (2022)
- [Killing Two Birds with One Stone: Quantization Achieves Privacy in Distributed Learning](https://mlsanity.com/paper/2304.13545/) (2023)
- [IMFL-AIGC: Incentive Mechanism Design for Federated Learning Empowered by Artificial Intelligence Generated Content](https://mlsanity.com/paper/2406.08526/) (2024)
- [Federated Learning Survey: A Multi-Level Taxonomy of Aggregation Techniques, Experimental Insights, and Future Frontiers](https://mlsanity.com/paper/2511.22616/) (2025)
- [Distributed Learning Meets 6G: A Communication and Computing Perspective](https://mlsanity.com/paper/2303.12802/) (2023)
- [Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework](https://mlsanity.com/paper/2406.11159/) (2024)
- [Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacy](https://mlsanity.com/paper/2408.04315/) (2024)
- [Federated Dynamic Spectrum Access](https://mlsanity.com/paper/2106.14976/) (2021)
- [Privacy-Preserving Ensemble Infused Enhanced Deep Neural Network Framework for Edge Cloud Convergence](https://mlsanity.com/paper/2305.09224/) (2023)
- [CRS-FL: Conditional Random Sampling for Communication-Efficient and Privacy-Preserving Federated Learning](https://mlsanity.com/paper/2306.00674/) (2023)
- [Traversal Learning: A Lossless And Efficient Distributed Learning Framework](https://mlsanity.com/paper/2504.07471/) (2025)
- [Accelerated Training of Federated Learning via Second-Order Methods](https://mlsanity.com/paper/2505.23588/) (2025)
- [Byzantine Outside, Curious Inside: Reconstructing Data Through Malicious Updates](https://mlsanity.com/paper/2506.11413/) (2025)
- [A Comparative Study of Federated Learning Aggregation Strategies under Homogeneous and Heterogeneous Data Distributions](https://mlsanity.com/paper/2605.11010/) (2026)

## Abstract (arXiv)

The performance of federated learning systems is bottlenecked by communication costs and training variance. The communication overhead problem is usually addressed by three communication-reduction techniques, namely, model compression, partial device participation, and periodic aggregation, at the cost of increased training variance. Different from traditional distributed learning systems, federated learning suffers from data heterogeneity (since the devices sample their data from possibly different distributions), which induces additional variance among devices during training. Various variance-reduced training algorithms have been introduced to combat the effects of data heterogeneity, while they usually cost additional communication resources to deliver necessary control information. Additionally, data privacy remains a critical issue in FL, and thus there have been attempts at bringing Differential Privacy to this framework as a mediator between utility and privacy requirements. This paper investigates the trade-offs between communication costs and training variance under a resource-constrained federated system theoretically and experimentally, and how communication reduction techniques interplay in a differentially private setting. The results provide important insights into designing practical privacy-aware federated learning systems.