A Horizon-Independent Regret Bound for Optimistic Hedge in General-Sum Games
Not on the map. Why?
Abstract (arXiv)
Can simple learning rules keep their regret bounded in self-play? Recent work achieves constant regret bounds through modified regularization and higher-order prediction. Yet for Optimistic Hedge, arguably the most canonical method in games, the best known individual regret bound remains logarithmic. In this work, we prove that plain Optimistic Hedge with a constant step size can attain On,d(1) individual regret in general-sum games with n players and d=(d1,ldots,dn) actions, under expected loss-vector feedback. As a corollary, its time-averaged play enjoys an On,d(1/T) coarse correlated equilibrium (CCE) gap. Our analysis represents Optimistic Hedge as a real-analytic recurrence on a compact space, which yields an exact finite-order difference relation that eliminates horizon dependence. Our proof hinges on nonconstructive Noetherianity argument of Frisch (1967), so the (n,d)-dependence remains implicit.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 45 references that are in the map. First posted on arXiv 19 Sep 2026.
Builds on
21 of 45 references.
References · 21
- 2024Self-Play Preference Optimization for Language Model Alignmentcited by 228
- 2023Nash Learning from Human Feedbackcited by 220
- 2013Optimization, Learning, and Games with Predictable Sequencescited by 187
- 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 176
- 2015Fast Convergence of Regularized Learning in Gamescited by 149
- 2019A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approachcited by 142
- 2021Near-Optimal No-Regret Learning in General Gamescited by 85
- 2020Hedging in games: Faster convergence of external and swap regretscited by 57
- 2021Near-Optimal No-Regret Learning for Correlated Equilibria in Multi-Player General-Sum Gamescited by 43
- 2022Uncoupled Learning Dynamics with O(log T) Swap Regret in Multiplayer Gamescited by 35
- 2022Near-Optimal No-Regret Learning Dynamics for General Convex Gamescited by 25
- 2021Beyond Time-Average Convergence: Near-Optimal Uncoupled Online Learning via Clairvoyant Multiplicative Weights Updatecited by 20
9 more
- 2025Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?cited by 20
- 2025Faster Rates for No-Regret Learning in General Games via Cautious Optimismcited by 10
- 2025Cautious Optimism: A Meta-Algorithm for Near-Constant Regret in General Gamescited by 7
- 2025Multiplayer Nash Preference Optimizationcited by 7
- 2026Sublogarithmic Swap Regret in Multiplayer General-Sum Games via Hybrid Regularizationcited by 4
- 2026Constant Individual Regret in General Gamescited by 3
- 2026Optimal Alternating Regret for Online Learning and Gamescited by 2
- 2026Constant regret in general games via higher-order optimismcited by 2
- 2026A Logarithmic Regret Bound for Optimistic Hedge in General-Sum Gamescited by 1
Built on by
Not cited yet.