mlsanityData to 7 Oct 2026

Cycles in adversarial regularized learning

arXiv 1709.02738 ↗First posted 8 Sep 2017cs.GT cs.LGcited by 122

Not on the map. Why?

Abstract (arXiv)

Regularized learning is a fundamental technique in online optimization, machine learning and many other fields of computer science. A natural question that arises in these settings is how regularized learning algorithms behave when faced against each other. We study a natural formulation of this problem by coupling regularized learning dynamics in zero-sum games. We show that the system's behavior is Poincar'e recurrent, implying that almost every trajectory revisits any (arbitrarily small) neighborhood of its starting point infinitely often. This cycling behavior is robust to the agents' choice of regularization mechanism (each agent could be using a different regularizer), to positive-affine transformations of the agents' utilities, and it also persists in the case of networked competition, i.e., for zero-sum polymatrix games.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 26; 2023: 29; 2024: 16; 2025: 15; 2026: 7.

122 papers.

Sort:

By area:

  1. 2023Nash Learning from Human Feedbackcited by 220
  2. 2021Independent Policy Gradient Methods for Competitive Reinforcement Learningcited by 97
  3. 2021Global Convergence of Multi-Agent Policy Gradient in Markov Potential Gamescited by 89
  4. 2021Near-Optimal No-Regret Learning in General Gamescited by 85
  5. 2021Fast Policy Extragradient Methods for Competitive Games with Entropy Regularizationcited by 61
  6. 2021Sig-Wasserstein GANs for Time Series Generationcited by 60
  7. 2021Adaptive Learning in Continuous Games: Optimal Regret Bounds and Convergence to Nash Equilibriumcited by 51
  8. 2021Scaling up Mean Field Games with Online Mirror Descentcited by 29
  9. 2023Uncoupled and Convergent Learning in Two-Player Zero-Sum Markov Games with Bandit Feedbackcited by 28
  10. 2021Last-iterate Convergence in Extensive-Form Gamescited by 27
  11. 2021The Confluence of Networks, Games and Learningcited by 26
  12. 2021Survival of the strictest: Stable and unstable equilibria under regularized learning with partial informationcited by 22
38 more
  1. 2022A unified stochastic approximation framework for learning in gamescited by 22
  2. 2022The Power of Regularization in Solving Extensive-Form Gamescited by 21
  3. 2023Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problemscited by 21
  4. 2021Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationalitycited by 20
  5. 2021Beyond Time-Average Convergence: Near-Optimal Uncoupled Online Learning via Clairvoyant Multiplicative Weights Updatecited by 20
  6. 2022FairNeuron: Improving Deep Neural Network Fairness with Adversary Games on Selective Neuronscited by 20
  7. 2023Autobidders with Budget and ROI Constraints: Efficiency, Regret, and Pacing Dynamicscited by 20
  8. 2024Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignmentcited by 20
  9. 2021Policy Gradient Methods Find the Nash Equilibrium in N-player General-sum Linear-quadratic Gamescited by 19
  10. 2023Is Learning in Games Good for the Learners?cited by 19
  11. 2022Tight Last-Iterate Convergence of the Extragradient and the Optimistic Gradient Descent-Ascent Algorithm for Constrained Monotone Variational Inequalitiescited by 18
  12. 2021How and Why to Manipulate Your Own Agent: On the Incentives of Users of Learning Agentscited by 17
  13. 2024Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimizationcited by 15
  14. 2021Doubly Optimal No-Regret Online Learning in Strongly Monotone Games with Bandit Feedbackcited by 14
  15. 2022Learning Sparse Graphon Mean Field Gamescited by 14
  16. 2022An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Gamescited by 14
  17. 2023On the Convergence of No-Regret Learning Dynamics in Time-Varying Gamescited by 14
  18. 2024Fast Last-Iterate Convergence of Learning in Games Requires Forgetful Algorithmscited by 14
  19. 2022Alternating Mirror Descent for Constrained Min-Max Gamescited by 13
  20. 2022On the convergence of policy gradient methods to Nash equilibria in general stochastic gamescited by 13
  21. 2022Riemannian Hamiltonian methods for min-max optimization on manifoldscited by 12
  22. 2023Doubly Optimal No-Regret Learning in Monotone Gamescited by 12
  23. 2021Learning in Matrix Games can be Arbitrarily Complexcited by 11
  24. 2021Online Learning in Periodic Zero-Sum Gamescited by 10
  25. 2023Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Playcited by 10
  26. 2022On Convergence of Gradient Descent Ascent: A Tight Local Analysiscited by 9
  27. 2022Last-Iterate Convergence with Full and Noisy Feedback in Two-Player Zero-Sum Gamescited by 9
  28. 2023Can We Find Nash Equilibria at a Linear Rate in Markov Games?cited by 9
  29. 2023Adaptively Perturbed Mirror Descent for Learning in Gamescited by 9
  30. 2021Solving Min-Max Optimization with Hidden Structure via Gradient Descent Ascentcited by 8
  31. 2021Nash Convergence of Mean-Based Learning Algorithms in First-Price Auctionscited by 8
  32. 2021Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equationscited by 8
  33. 2022Mutation-Driven Follow the Regularized Leader for Last-Iterate Convergence in Zero-Sum Gamescited by 8
  34. 2022Nash Equilibria and Pitfalls of Adversarial Training in Adversarial Robustness Gamescited by 8
  35. 2023Multi-Player Zero-Sum Markov Games with Networked Separable Interactionscited by 8
  36. 2024COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferencescited by 8
  37. 2023Asymptotic Convergence and Performance of Multi-Agent Q-Learning Dynamicscited by 6
  38. 2023Adaptive, Doubly Optimal No-Regret Learning in Strongly Monotone and Exp-Concave Games with Gradient Feedbackcited by 6

50 most cited of 122 All 122 (JSON)