mlsanityData to 14 Oct 2026

Distributed stochastic proximal algorithm with random reshuffling for non-smooth finite-sum optimization

arXiv 2111.03820 ↗First posted 6 Nov 2021math.OC cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

The non-smooth finite-sum minimization is a fundamental problem in machine learning. This paper develops a distributed stochastic proximal-gradient algorithm with random reshuffling to solve the finite-sum minimization over time-varying multi-agent networks. The objective function is a sum of differentiable convex functions and non-smooth regularization. Each agent in the network updates local variables with a constant step-size by local information and cooperates to seek an optimal solution. We prove that local variable estimates generated by the proposed algorithm achieve consensus and are attracted to a neighborhood of the optimal solution in expectation with an O(frac1T+frac1√T) convergence rate, where T is the total number of iterations. Finally, some comparative simulations are provided to verify the convergence performance of the proposed algorithm.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 26 of its 61 references that are in the map. First posted on arXiv 6 Nov 2021.

Builds on

26 of 61 references.

References · 26

  1. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  2. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  3. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  4. 2020Random Reshuffling: Simple Analysis with Vast Improvementscited by 115
  5. 2016NEXT: In-Network Nonconvex Optimizationcited by 107
  6. 2015Why Random Reshuffling Beats Stochastic Gradient Descentcited by 79
  7. 2014A Proximal Stochastic Gradient Method with Progressive Variance Reductioncited by 78
  8. 2019SGD without Replacement: Sharper Rates for General Smooth Convex Functionscited by 61
  9. 2020SGD with shuffling: optimal rates without component convexity and large epoch requirementscited by 58
  10. 2015Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Surveycited by 45
  11. 2019The Complexity of Finding Stationary Points with Stochastic Gradient Descentcited by 35
  12. 2018A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimizationcited by 33
14 more
  1. 2019Variance-Reduced Decentralized Stochastic Optimization with Accelerated Convergencecited by 33
  2. 2021Proximal and Federated Random Reshufflingcited by 33
  3. 2017Convergence Analysis of Distributed Stochastic Gradient Descent with Shufflingcited by 23
  4. 2018Robust Asynchronous Stochastic Gradient-Push: Asymptotically Optimal and Network-Independent Performance for Strongly Convex Functionscited by 20
  5. 2019Decentralized Proximal Gradient Algorithms with Linear Convergence Ratescited by 19
  6. 2020The Strength of Nesterov's Extrapolation in the Individual Convergence of Nonsmooth Optimizationcited by 9
  7. 2020Nonsmoothness in Machine Learning: specific structure, proximal identification, and applicationscited by 9
  8. 2017Improving Deep Learning by Inverse Square Root Linear Units (ISRLUs)cited by 7
  9. 2020Push-SAGA: A decentralized stochastic algorithm with variance reduction over directed graphscited by 7
  10. 2019Communication-Censored Distributed Stochastic Gradient Descentcited by 4
  11. 2016Asymptotic Properties of Primal-Dual Algorithm for Distributed Stochastic Optimization Over Random Networkscited by 3
  12. 2017Doubly Accelerated Stochastic Variance Reduced Dual Averaging Method for Regularized Empirical Risk Minimizationcited by 2
  13. 2019Distributed Proximal Algorithms for Multi-Agent Optimization with Coupled Inequality Constraintscited by 2
  14. 2021Distributed proximal gradient algorithm for non-smooth non-convex optimization over time-varying networkscited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 1.

2 papers.

  1. 2021Distributed Random Reshuffling over Networkscited by 7
  2. 2023Distributed Random Reshuffling Methods with Improved Convergencecited by 2