# Understanding Iterative Combinatorial Auction Designs via Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2402.19420 · HTML: https://mlsanity.com/paper/2402.19420/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 66 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning](https://mlsanity.com/paper/2206.15378/) (2022)
- [Solving Large Imperfect Information Games Using CFR+](https://mlsanity.com/paper/1407.5042/) (2014)
- [Deep Models of Interactions Across Sets](https://mlsanity.com/paper/1803.02879/) (2018)
- [Solving Imperfect-Information Games via Discounted Regret Minimization](https://mlsanity.com/paper/1809.04040/) (2018)
- [Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1802.09756/) (2018)
- [No-regret learning and mixed Nash equilibria: They do not mix](https://mlsanity.com/paper/2010.09514/) (2020)
- [Optimal Auctions through Deep Learning: Advances in Differentiable Economics](https://mlsanity.com/paper/1706.03459/) (2017)
- [Learning to Play No-Press Diplomacy with Best Response Policy Iteration](https://mlsanity.com/paper/2006.04635/) (2020)
- [Hindsight and Sequential Rationality of Correlated Play](https://mlsanity.com/paper/2012.05874/) (2020)
- [Artificial Intelligence and Auction Design](https://mlsanity.com/paper/2202.05947/) (2022)
- [Computing Bayes Nash Equilibrium Strategies in Auction Games via Simultaneous Online Dual Averaging](https://mlsanity.com/paper/2208.02036/) (2022)
- [Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learning](https://mlsanity.com/paper/2302.00797/) (2023)
- [Computing Bayes-Nash Equilibria in Combinatorial Auctions with Verification](https://mlsanity.com/paper/1812.01955/) (2018)
- [Platform Behavior under Market Shocks: A Simulation Framework and Reinforcement-Learning Based Study](https://mlsanity.com/paper/2203.13395/) (2022)
- [Computing Perfect Bayesian Equilibria in Sequential Auctions with Verification](https://mlsanity.com/paper/2312.04516/) (2023)
- [Computational Analysis of Perfect-Information Position Auctions](https://mlsanity.com/paper/1408.0703/) (2014)
- [The Cost of Simple Bidding in Combinatorial Auctions](https://mlsanity.com/paper/2011.12237/) (2020)
- [A Reinforcement Learning Approach in Multi-Phase Second-Price Auction Design](https://mlsanity.com/paper/2210.10278/) (2022)
- [Equilibrium Computation in Multi-Stage Auctions and Contests](https://mlsanity.com/paper/2312.11751/) (2023)

## Built on by (most cited first, 2 of 2)

- [Prices, Bids, Values: One ML-Powered Combinatorial Auction to Rule Them All](https://mlsanity.com/paper/2411.09355/) (2024)
- [Does a Toehold Make a Bidder Bolder? Preemption and Multiplicity in Multi-Round Takeover Auctions](https://mlsanity.com/paper/2608.08407/) (2026)

## Abstract (arXiv)

Iterative combinatorial auctions are widely used in high stakes settings such as spectrum auctions. Such auctions can be hard to analyze, making it difficult for bidders to determine how to behave and for designers to optimize auction rules to ensure desirable outcomes such as high revenue or welfare. In this paper, we investigate whether multi-agent reinforcement learning (MARL) algorithms can be used to understand iterative combinatorial auctions, given that these algorithms have recently shown empirical success in several other domains. We find that MARL can indeed benefit auction analysis, but that deploying it effectively is nontrivial. We begin by describing modelling decisions that keep the resulting game tractable without sacrificing important features such as imperfect information or asymmetry between bidders. We also discuss how to navigate pitfalls of various MARL algorithms, how to overcome challenges in verifying convergence, and how to generate and interpret multiple equilibria. We illustrate the promise of our resulting approach by using it to evaluate a specific rule change to a clock auction, finding substantially different auction outcomes due to complex changes in bidders' behavior.