# Modelling Bounded Rationality in Multi-Agent Interactions by Generalized Recursive Reasoning

arXiv: https://arxiv.org/abs/1901.09216 · HTML: https://mlsanity.com/paper/1901.09216/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2019.

## Built on by (most cited first, 24 of 24)

- [Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2109.11251/) (2021)
- [Multi-Agent Reinforcement Learning is a Sequence Modeling Problem](https://mlsanity.com/paper/2205.14953/) (2022)
- [CodeR: Issue Resolving with Multi-Agent and Task Graphs](https://mlsanity.com/paper/2406.01304/) (2024)
- [Heterogeneous-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.09870/) (2023)
- [Settling the Variance of Multi-Agent Policy Gradients](https://mlsanity.com/paper/2108.08612/) (2021)
- [Multi-Agent Constrained Policy Optimisation](https://mlsanity.com/paper/2110.02793/) (2021)
- [Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks](https://mlsanity.com/paper/2112.02845/) (2021)
- [Model-Based Opponent Modeling](https://mlsanity.com/paper/2108.01843/) (2021)
- [Maximum Entropy Heterogeneous-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.10715/) (2023)
- [Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teaming](https://mlsanity.com/paper/2201.08484/) (2022)
- [A Game-Theoretic Approach to Multi-Agent Trust Region Optimization](https://mlsanity.com/paper/2106.06828/) (2021)
- [Understanding Value Decomposition Algorithms in Deep Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2202.04868/) (2022)
- [(Ir)rationality in AI: State of the Art, Research Challenges and Open Questions](https://mlsanity.com/paper/2311.17165/) (2023)
- [Bounded rationality for relaxing best response and mutual consistency: The Quantal Hierarchy model of decision-making](https://mlsanity.com/paper/2106.15844/) (2021)
- [Solving Dynamic Principal-Agent Problems with a Rationally Inattentive Principal](https://mlsanity.com/paper/2202.01691/) (2022)
- [Recursive Reasoning Graph for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2203.02844/) (2022)
- [Decision-making with Speculative Opponent Models](https://mlsanity.com/paper/2211.11940/) (2022)
- [Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.00787/) (2024)
- [Social Environment Design](https://mlsanity.com/paper/2402.14090/) (2024)
- [Active Legibility in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2410.20954/) (2024)
- [A Computable Game-Theoretic Framework for Multi-Agent Theory of Mind](https://mlsanity.com/paper/2511.22536/) (2025)
- [Cognitive Level-k Meta-Learning for Safe and Pedestrian-Aware Autonomous Driving](https://mlsanity.com/paper/2212.08800/) (2022)
- [Off-Policy Action Anticipation in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.01447/) (2023)
- [Vairiational Stochastic Games](https://mlsanity.com/paper/2503.06037/) (2025)

## Abstract (arXiv)

Though limited in real-world decision making, most multi-agent reinforcement learning (MARL) models assume perfectly rational agents -- a property hardly met due to individual's cognitive limitation and/or the tractability of the decision problem. In this paper, we introduce generalized recursive reasoning (GR2) as a novel framework to model agents with different hierarchical levels of rationality; our framework enables agents to exhibit varying levels of "thinking" ability thereby allowing higher-level agents to best respond to various less sophisticated learners. We contribute both theoretically and empirically. On the theory side, we devise the hierarchical framework of GR2 through probabilistic graphical models and prove the existence of a perfect Bayesian equilibrium. Within the GR2, we propose a practical actor-critic solver, and demonstrate its convergent property to a stationary point in two-player games through Lyapunov analysis. On the empirical side, we validate our findings on a variety of MARL benchmarks. Precisely, we first illustrate the hierarchical thinking process on the Keynes Beauty Contest, and then demonstrate significant improvements compared to state-of-the-art opponent modeling baselines on the normal-form games and the cooperative navigation benchmark.