mlsanityData to 14 Oct 2026

Stochastic Unrolled Federated Learning

arXiv 2305.15371 ↗First posted 24 May 2023cs.LG eess.SPcited by 6

Not on the map. Why?

Abstract (arXiv)

Algorithm unrolling has emerged as a learning-based optimization paradigm that unfolds truncated iterative algorithms in trainable neural-network optimizers. We introduce Stochastic UnRolled Federated learning (SURF), a method that expands algorithm unrolling to federated learning in order to expedite its convergence. Our proposed method tackles two challenges of this expansion, namely the need to feed whole datasets to the unrolled optimizers to find a descent direction and the decentralized nature of federated learning. We circumvent the former challenge by feeding stochastic mini-batches to each unrolled layer and imposing descent constraints to guarantee its convergence. We address the latter challenge by unfolding the distributed gradient descent (DGD) algorithm in a graph neural network (GNN)-based unrolled architecture, which preserves the decentralized nature of training in federated learning. We theoretically prove that our proposed unrolled optimizer converges to a near-optimal region infinitely often. Through extensive numerical experiments, we also demonstrate the effectiveness of the proposed framework in collaborative training of image classifiers.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 56 of its 88 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

56 of 88 references.

References · 56

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2016Deep Learning with Differential Privacycited by 2,728
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  7. 2021Model-Contrastive Federated Learningcited by 766
  8. 2016Learning to learn by gradient descent by gradient descentcited by 620
  9. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  10. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 421
  11. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  12. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
38 more
  1. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 315
  2. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
  3. 2017Asynchronous Decentralized Parallel Stochastic Gradient Descentcited by 159
  4. 2019Stability Properties of Graph Neural Networkscited by 127
  5. 2020Graphon Neural Networks and the Transferability of Graph Neural Networkscited by 118
  6. 2015Asynchronous Parallel Stochastic Gradient for Nonconvex Optimizationcited by 116
  7. 2021Decentralized Federated Averagingcited by 114
  8. 2017Learned Optimizers that Scale and Generalizecited by 110
  9. 2018Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholdscited by 85
  10. 2019Learning to Optimize Multigrid PDE Solverscited by 79
  11. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 76
  12. 2020Graphs, Convolutions, and Neural Networks: From Graph Filters to Graph Neural Networkscited by 75
  13. 2020Unfolding WMMSE using Graph Neural Networks for Efficient Power Allocationcited by 61
  14. 2013On the Linear Convergence of the ADMM in Decentralized Consensus Optimizationcited by 57
  15. 2021Decentralized Federated Learning through Proxy Model Sharingcited by 57
  16. 2021Decentralized Federated Learning: Balancing Communication and Computing Costscited by 55
  17. 2017Learning Gradient Descent: Better Generalization and Longer Horizonscited by 50
  18. 2019Local Differential Privacy for Deep Learningcited by 46
  19. 2020Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systemscited by 44
  20. 2021Constrained Learning with Non-Convex Lossescited by 42
  21. 2021Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learningcited by 39
  22. 2019Stability of Graph Scattering Transformscited by 34
  23. 2020Graph Unrolling Networks: Interpretable Neural Networks for Graph Signal Denoisingcited by 33
  24. 2022RU-Net: Regularized Unrolling Network for Scene Graph Generationcited by 28
  25. 2021Learning to Learn Graph Topologiescited by 26
  26. 2020Safeguarded Learned Convex Optimizationcited by 24
  27. 2019Learning to Optimize in Swarmscited by 23
  28. 2019Model Learning: Primal Dual Networks for Fast MR imagingcited by 22
  29. 2020Improved Adversarial Training via Learned Optimizercited by 22
  30. 2022How Much Privacy Does Federated Learning with Secure Aggregation Guarantee?cited by 19
  31. 2021SGD-Net: Efficient Model-Based Deep Learning with Theoretical Guaranteescited by 18
  32. 2018Learning to Defend by Learning to Attackcited by 17
  33. 2020Iterative Amortized Policy Optimizationcited by 13
  34. 2016Tradeoffs between Convergence Speed and Reconstruction Accuracy in Inverse Problemscited by 12
  35. 2019Controlling Neural Networks via Energy Dissipationcited by 12
  36. 2021Space-Time Graph Neural Networkscited by 10
  37. 2020A deep primal-dual proximal network for image restorationcited by 9
  38. 2022Coordinated Sum-Rate Maximization in Multicell MU-MIMO with Deep Unrollingcited by 8

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 0; 2025: 1; 2026: 2.

6 papers.

  1. 2023Robust Stochastically-Descending Unrolled Networkscited by 9
  2. 2023Limited Communications Distributed Optimization via Deep Unfolded Distributed ADMMcited by 2
  3. 2025Unrolled Graph Neural Networks for Constrained Optimizationcited by 1
  4. 2023Improving Federated Aggregation with Deep Unfolding Networkscited by 0
  5. 2026A Constrained Optimization Perspective of Unrolled Transformerscited by 0
  6. 2026Unrolled Neural Networks for Constrained Optimizationcited by 0