# R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games

arXiv: https://arxiv.org/abs/2006.16679 · HTML: https://mlsanity.com/paper/2006.16679/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2020.

## Built on by (most cited first, 11 of 11)

- [Deep Multiagent Reinforcement Learning: Challenges and Directions](https://mlsanity.com/paper/2106.15691/) (2021)
- [Differentially Private Federated Bayesian Optimization with Distributed Exploration](https://mlsanity.com/paper/2110.14153/) (2021)
- [Sample-Then-Optimize Batch Neural Thompson Sampling](https://mlsanity.com/paper/2210.06850/) (2022)
- [Model-Based Opponent Modeling](https://mlsanity.com/paper/2108.01843/) (2021)
- [Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search](https://mlsanity.com/paper/2201.09785/) (2022)
- [Bayesian Optimization under Stochastic Delayed Feedback](https://mlsanity.com/paper/2206.09341/) (2022)
- [Trusted-Maximizers Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/2107.14465/) (2021)
- [On Provably Robust Meta-Bayesian Optimization](https://mlsanity.com/paper/2206.06872/) (2022)
- [Training-Free Neural Active Learning with Initialization-Robustness Guarantees](https://mlsanity.com/paper/2306.04454/) (2023)
- [Convolutional Normalizing Flows for Deep Gaussian Processes](https://mlsanity.com/paper/2104.08472/) (2021)
- [Bidirectional Distillation: A Mixed-Play Framework for Multi-Agent Generalizable Behaviors](https://mlsanity.com/paper/2505.11100/) (2025)

## Abstract (arXiv)

This paper presents a recursive reasoning formalism of Bayesian optimization (BO) to model the reasoning process in the interactions between boundedly rational, self-interested agents with unknown, complex, and costly-to-evaluate payoff functions in repeated games, which we call Recursive Reasoning-Based BO (R2-B2). Our R2-B2 algorithm is general in that it does not constrain the relationship among the payoff functions of different agents and can thus be applied to various types of games such as constant-sum, general-sum, and common-payoff games. We prove that by reasoning at level 2 or more and at one level higher than the other agents, our R2-B2 agent can achieve faster asymptotic convergence to no regret than that without utilizing recursive reasoning. We also propose a computationally cheaper variant of R2-B2 called R2-B2-Lite at the expense of a weaker convergence guarantee. The performance and generality of our R2-B2 algorithm are empirically demonstrated using synthetic games, adversarial machine learning, and multi-agent reinforcement learning.