mlsanityData to 7 Oct 2026

Private Non-Convex Federated Learning Without a Trusted Server

arXiv 2203.06735 ↗First posted 13 Mar 2022cs.LG cs.CR math.OCcited by 26

Not on the map. Why?

Abstract (arXiv)

We study federated learning (FL) -- especially cross-silo FL -- with non-convex loss functions and data from people who do not trust the server or other silos. In this setting, each silo (e.g. hospital) must protect the privacy of each person's data (e.g. patient's medical record), even if the server or other silos act as adversarial eavesdroppers. To that end, we consider inter-silo record-level (ISRL) differential privacy (DP), which requires silo~i's communications to satisfy record/item-level DP. We propose novel ISRL-DP algorithms for FL with heterogeneous (non-i.i.d.) silo data and two classes of Lipschitz continuous loss functions: First, we consider losses satisfying the Proximal Polyak-Lojasiewicz (PL) inequality, which is an extension of the classical PL condition to the constrained setting. In contrast to our result, prior works only considered unconstrained private optimization with Lipschitz PL loss, which rules out most interesting PL losses such as strongly convex problems and linear/logistic regression. Our algorithms nearly attain the optimal strongly convex, homogeneous (i.i.d.) rate for ISRL-DP FL without assuming convexity or i.i.d. data. Second, we give the first private algorithms for non-convex non-smooth loss functions. Our utility bounds even improve on the state-of-the-art bounds for smooth losses. We complement our upper bounds with lower bounds. Additionally, we provide shuffle DP (SDP) algorithms that improve over the state-of-the-art central DP algorithms under more practical trust assumptions. Numerical experiments show that our algorithm has better accuracy than baselines for most privacy levels. All the codes are publicly available at: https://github.com/ghafeleb/Private-NonConvex-Federated-Learning-Without-a-Trusted-Server.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 59 of its 110 references that are in the map. First posted on arXiv 13 Mar 2022.

Builds on

59 of 110 references.

References · 59

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2014Convex Optimization: Algorithms and Complexitycited by 569
  8. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  9. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  10. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  11. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  12. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
38 more
  1. 2019Lower Bounds for Non-Convex Stochastic Optimizationcited by 303
  2. 2013The Composition Theorem for Differential Privacycited by 290
  3. 2018SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimatorcited by 283
  4. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 247
  5. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 231
  6. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  7. 2019Private Stochastic Convex Optimization with Optimal Ratescited by 183
  8. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 163
  9. 2020Private Stochastic Convex Optimization: Optimal Rates in Linear Timecited by 152
  10. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  11. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  12. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  13. 2016Concentrated Differential Privacycited by 139
  14. 2020LDP-Fed: Federated Learning with Local Differential Privacycited by 129
  15. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 127
  16. 2018Distributed Differential Privacy via Shufflingcited by 115
  17. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 104
  18. 2021Hyperparameter Tuning with Renyi Differential Privacycited by 104
  19. 2020Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shufflingcited by 96
  20. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  21. 2018Private Selection from Private Candidatescited by 75
  22. 2020Local Differential Privacy based Federated Learning for Internet of Thingscited by 74
  23. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  24. 2021Learning with User-Level Privacycited by 72
  25. 2017Efficient Private ERM for Smooth Objectivescited by 61
  26. 2017Non-convex Finite-Sum Optimization Via SCSG Methodscited by 61
  27. 2017Prochlo: Strong Privacy for Analytics in the Crowdcited by 51
  28. 2020Wireless Federated Learning with Local Differential Privacycited by 50
  29. 2019Local Differential Privacy for Deep Learningcited by 46
  30. 2021Differentially Private Stochastic Optimization: New Results in Convex and Non-Convex Settingscited by 45
  31. 2018DP-ADMM: ADMM-based Distributed Learning with Differential Privacycited by 41
  32. 2020Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluationcited by 39
  33. 2019The Complexity of Making the Gradient Small in Stochastic Convex Optimizationcited by 33
  34. 2021Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Lossescited by 30
  35. 2019Efficient Privacy-Preserving Stochastic Nonconvex Optimizationcited by 27
  36. 2020FLAME: Differentially Private Federated Learning in the Shuffle Modelcited by 27
  37. 2020Private Stochastic Non-Convex Optimization: Adaptive Algorithms and Tighter Generalization Boundscited by 26
  38. 2020Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimizationcited by 25

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 6; 2024: 8; 2025: 8; 2026: 1.

26 papers.

Sort:

By area:

  1. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  2. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  3. 2022Stochastic Differentially Private and Fair Learningcited by 16
  4. 2023On Differentially Private Federated Linear Contextual Banditscited by 14
  5. 2024FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneitycited by 13
  6. 2024How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimizationcited by 10
  7. 2023Optimal Differentially Private Model Training with Public Datacited by 9
  8. 2023ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacycited by 9
  9. 2024Private Heterogeneous Federated Learning Without a Trusted Server Revisited: Error-Optimal and Communication-Efficient Algorithms for Convex Lossescited by 8
  10. 2024Cross-silo Federated Learning with Record-level Personalized Differential Privacycited by 7
  11. 2023Differentially Private Non-Convex Optimization under the KL Condition with Optimal Ratescited by 4
  12. 2023Toward the Tradeoffs between Privacy, Fairness and Utility in Federated Learningcited by 4
14 more
  1. 2024PrivSGP-VR: Differentially Private Variance-Reduced Stochastic Gradient Push with Tight Utility Boundscited by 4
  2. 2022Private Stochastic Optimization With Large Worst-Case Lipschitz Parametercited by 3
  3. 2024Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systemscited by 3
  4. 2025Second-Order Convergence in Private Stochastic Non-Convex Optimizationcited by 3
  5. 2025Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practicecited by 3
  6. 2025Smoothed Normalization for Efficient Distributed Private Optimizationcited by 1
  7. 2025Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellationcited by 1
  8. 2025DP-CSGP: Differentially Private Stochastic Gradient Push with Compressed Communicationcited by 1
  9. 2024Convergent Differential Privacy Analysis for General Federated Learningcited by 0
  10. 2024A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Datacited by 0
  11. 2025Personalized Federated Training of Diffusion Models with Privacy Guaranteescited by 0
  12. 2025On the Inherent Privacy of Zeroth Order Projected Gradient Descentcited by 0
  13. 2025First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptionscited by 0
  14. 2026ERIS: Enhancing Privacy and Scalability in Federated Learning via Federated Shard Aggregationcited by 0