# Solving Games with Functional Regret Estimation

arXiv: https://arxiv.org/abs/1411.7974 · HTML: https://mlsanity.com/paper/1411.7974/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2014.

## Built on by (most cited first, 12 of 12)

- [Near-Optimal Learning of Extensive-Form Games with Imperfect Information](https://mlsanity.com/paper/2202.01752/) (2022)
- [A Survey on Self-play Methods in Reinforcement Learning](https://mlsanity.com/paper/2408.01072/) (2024)
- [Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games](https://mlsanity.com/paper/2102.06973/) (2021)
- [PerfectDou: Dominating DouDizhu with Perfect Information Distillation](https://mlsanity.com/paper/2203.16406/) (2022)
- [RL-CFR: Improving Action Abstraction for Imperfect Information Extensive-Form Games with Reinforcement Learning](https://mlsanity.com/paper/2403.04344/) (2024)
- [Equivalence Analysis between Counterfactual Regret Minimization and Online Mirror Descent](https://mlsanity.com/paper/2110.04961/) (2021)
- [Hierarchical Deep Counterfactual Regret Minimization](https://mlsanity.com/paper/2305.17327/) (2023)
- [Learning to Be Cautious](https://mlsanity.com/paper/2110.15907/) (2021)
- [Multi-agent Reinforcement Learning in OpenSpiel: A Reproduction Report](https://mlsanity.com/paper/2103.00187/) (2021)
- [Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games: Corrections](https://mlsanity.com/paper/2205.12031/) (2022)
- [Coachable agents for interactive gameplay](https://mlsanity.com/paper/2607.00642/) (2026)
- [CFR without Unbiasedness: Deterministic Guarantees for Persistent Public-Chance Schedules](https://mlsanity.com/paper/2608.14761/) (2026)

## Abstract (arXiv)

We propose a novel online learning method for minimizing regret in large extensive-form games. The approach learns a function approximator online to estimate the regret for choosing a particular action. A no-regret algorithm uses these estimates in place of the true regrets to define a sequence of policies. We prove the approach sound by providing a bound relating the quality of the function approximation and regret of the algorithm. A corollary being that the method is guaranteed to converge to a Nash equilibrium in self-play so long as the regrets are ultimately realizable by the function approximator. Our technique can be understood as a principled generalization of existing work on abstraction in large games; in our work, both the abstraction as well as the equilibrium are learned during self-play. We demonstrate empirically the method achieves higher quality strategies than state-of-the-art abstraction techniques given the same resources.