# FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection

arXiv: https://arxiv.org/abs/2003.10637 · HTML: https://mlsanity.com/paper/2003.10637/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2020.

## Built on by (most cited first, 30 of 36)

- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [Federated Learning with Local Differential Privacy: Trade-offs between Privacy, Utility, and Communication](https://mlsanity.com/paper/2102.04737/) (2021)
- [Joint Privacy Enhancement and Quantization in Federated Learning](https://mlsanity.com/paper/2208.10888/) (2022)
- [Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacy](https://mlsanity.com/paper/2202.07178/) (2022)
- [Stronger Privacy for Federated Collaborative Filtering with Implicit Feedback](https://mlsanity.com/paper/2105.03941/) (2021)
- [A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacy](https://mlsanity.com/paper/2302.10637/) (2023)
- [Graph-Fraudster: Adversarial Attacks on Graph Neural Network Based Vertical Federated Learning](https://mlsanity.com/paper/2110.06468/) (2021)
- [Transparent Contribution Evaluation for Secure Federated Learning on Blockchain](https://mlsanity.com/paper/2101.10572/) (2021)
- [DPD-fVAE: Synthetic Data Generation Using Federated Variational Autoencoders With Differentially-Private Decoder](https://mlsanity.com/paper/2211.11591/) (2022)
- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [FL-Market: Trading Private Models in Federated Learning](https://mlsanity.com/paper/2106.04384/) (2021)
- [Efficient and Reliable Overlay Networks for Decentralized Federated Learning](https://mlsanity.com/paper/2112.15486/) (2021)
- [Improving Differentially Private SGD via Randomly Sparsified Gradients](https://mlsanity.com/paper/2112.00845/) (2021)
- [Constrained Differentially Private Federated Learning for Low-bandwidth Devices](https://mlsanity.com/paper/2103.00342/) (2021)
- [Wireless Federated Learning with Limited Communication and Differential Privacy](https://mlsanity.com/paper/2106.00564/) (2021)
- [FLSys: Toward an Open Ecosystem for Federated Learning Mobile Apps](https://mlsanity.com/paper/2111.09445/) (2021)
- [Task-aware Privacy Preservation for Multi-dimensional Data](https://mlsanity.com/paper/2110.02329/) (2021)
- [Communication and Energy Efficient Wireless Federated Learning with Intrinsic Privacy](https://mlsanity.com/paper/2304.07460/) (2023)
- [Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy](https://mlsanity.com/paper/2305.00873/) (2023)
- [OLIVE: Oblivious Federated Learning on Trusted Execution Environment against the risk of sparsification](https://mlsanity.com/paper/2202.07165/) (2022)
- [FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learning](https://mlsanity.com/paper/2303.05206/) (2023)
- [FedPerm: Private and Robust Federated Learning by Parameter Permutation](https://mlsanity.com/paper/2208.07922/) (2022)
- [Review of Mathematical Optimization in Federated Learning](https://mlsanity.com/paper/2412.01630/) (2024)
- [On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times](https://mlsanity.com/paper/2101.04163/) (2021)
- [Lightweight Transformer in Federated Setting for Human Activity Recognition](https://mlsanity.com/paper/2110.00244/) (2021)
- [Optimizing the Numbers of Queries and Replies in Federated Learning with Differential Privacy](https://mlsanity.com/paper/2107.01895/) (2021)
- [Beyond Statistical Estimation: Differentially Private Individual Computation via Shuffling](https://mlsanity.com/paper/2406.18145/) (2024)
- [AdaDPIGU: Differentially Private SGD with Adaptive Clipping and Importance-Based Gradient Updates for Deep Neural Networks](https://mlsanity.com/paper/2507.06525/) (2025)
- [Understanding the Interplay between Privacy and Robustness in Federated Learning](https://mlsanity.com/paper/2106.07033/) (2021)
- [sqSGD: Locally Private and Communication Efficient Federated Learning](https://mlsanity.com/paper/2206.10565/) (2022)

## Abstract (arXiv)

As massive data are produced from small gadgets, federated learning on mobile devices has become an emerging trend. In the federated setting, Stochastic Gradient Descent (SGD) has been widely used in federated learning for various machine learning models. To prevent privacy leakages from gradients that are calculated on users' sensitive data, local differential privacy (LDP) has been considered as a privacy guarantee in federated SGD recently. However, the existing solutions have a dimension dependency problem: the injected noise is substantially proportional to the dimension d. In this work, we propose a two-stage framework FedSel for federated SGD under LDP to relieve this problem. Our key idea is that not all dimensions are equally important so that we privately select Top-k dimensions according to their contributions in each iteration of federated SGD. Specifically, we propose three private dimension selection mechanisms and adapt the gradient accumulation technique to stabilize the learning process with noisy updates. We also theoretically analyze privacy, accuracy and time complexity of FedSel, which outperforms the state-of-the-art solutions. Experiments on real-world and synthetic datasets verify the effectiveness and efficiency of our framework.