mlsanityData to 7 Oct 2026

Accelerated Multiplicative Weights Update Avoids Saddle Points almost always

arXiv 2204.11407 ↗First posted 25 Apr 2022math.OC cs.LG cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider non-convex optimization problems with constraint that is a product of simplices. A commonly used algorithm in solving this type of problem is the Multiplicative Weights Update (MWU), an algorithm that is widely used in game theory, machine learning and multi-agent systems. Despite it has been known that MWU avoids saddle points, there is a question that remains unaddressed:"Is there an accelerated version of MWU that avoids saddle points provably?" In this paper we provide a positive answer to above question. We provide an accelerated MWU based on Riemannian Accelerated Gradient Descent, and prove that the Riemannian Accelerated Gradient Descent, thus the accelerated MWU, almost always avoid saddle points.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 21 references that are in the map. First posted on arXiv 25 Apr 2022.

Builds on

11 of 21 references.

References · 11

  1. 2017How to Escape Saddle Points Efficientlycited by 284
  2. 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 241
  3. 2017Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descentcited by 66
  4. 2019First-order methods almost always avoid saddle points: the case of vanishing step-sizescited by 35
  5. 2016Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regionscited by 33
  6. 2019Escaping from saddle points on Riemannian manifoldscited by 33
  7. 2019Efficiently escaping saddle points on manifoldscited by 24
  8. 2019Heavy-ball Algorithms Always Escape Saddle Pointscited by 13
  9. 2018Towards Riemannian Accelerated Gradient Methodscited by 12
  10. 2018Multiplicative Weights Update as a Distributed Constrained Optimization Algorithm: Convergence to Second-order Stationary Points Almost Alwayscited by 7
  11. 2019SNAP: Finding Approximate Second-Order Stationary Solutions Efficiently for Non-convex Linearly Constrained Problemscited by 2

Built on by

Not cited yet.