# Games with Payments between Learning Agents

arXiv: https://arxiv.org/abs/2405.20880 · HTML: https://mlsanity.com/paper/2405.20880/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 111 references that are in the map.
- First posted on arXiv 31 May 2024.

## Builds on (most cited first, 30 of 43)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Multi-agent Reinforcement Learning in Sequential Social Dilemmas](https://mlsanity.com/paper/1702.03037/) (2017)
- [Fast Convergence of Regularized Learning in Games](https://mlsanity.com/paper/1507.00407/) (2015)
- [Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimization](https://mlsanity.com/paper/1807.04252/) (2018)
- [Near-Optimal No-Regret Learning in General Games](https://mlsanity.com/paper/2108.06924/) (2021)
- [Algorithmic Collusion by Large Language Models](https://mlsanity.com/paper/2404.00806/) (2024)
- [Learning to Incentivize Other Learning Agents](https://mlsanity.com/paper/2006.06051/) (2020)
- [Strategizing against No-regret Learners](https://mlsanity.com/paper/1909.13861/) (2019)
- [Selling to a No-Regret Buyer](https://mlsanity.com/paper/1711.09176/) (2017)
- [The Sample Complexity of Online Contract Design](https://mlsanity.com/paper/2211.05732/) (2022)
- [Strategizing against Learners in Bayesian Games](https://mlsanity.com/paper/2205.08562/) (2022)
- [Learning Reciprocity in Complex Sequential Social Dilemmas](https://mlsanity.com/paper/1903.08082/) (2019)
- [Econometrics for Learning Agents](https://mlsanity.com/paper/1505.00720/) (2015)
- [Learning in Auctions: Regret is Hard, Envy is Easy](https://mlsanity.com/paper/1511.01411/) (2015)
- [Contracting with a Learning Agent](https://mlsanity.com/paper/2401.16198/) (2024)
- [Auctions Between Regret-Minimizing Agents](https://mlsanity.com/paper/2110.11855/) (2021)
- [Autobidders with Budget and ROI Constraints: Efficiency, Regret, and Pacing Dynamics](https://mlsanity.com/paper/2301.13306/) (2023)
- [Towards Efficient Auctions in an Auto-bidding World](https://mlsanity.com/paper/2103.13356/) (2021)
- [Selling to Multiple No-Regret Buyers](https://mlsanity.com/paper/2307.04175/) (2023)
- [How and Why to Manipulate Your Own Agent: On the Incentives of Users of Learning Agents](https://mlsanity.com/paper/2112.07640/) (2021)
- [Pareto-Optimal Algorithms for Learning in Games](https://mlsanity.com/paper/2402.09549/) (2024)
- [Bid Prediction in Repeated Auctions with Learning](https://mlsanity.com/paper/2007.13193/) (2020)
- [Convergence Analysis of No-Regret Bidding Algorithms in Repeated Auctions](https://mlsanity.com/paper/2009.06136/) (2020)
- [Principal-Agent Reward Shaping in MDPs](https://mlsanity.com/paper/2401.00298/) (2023)
- [The Price of Anarchy in Auctions](https://mlsanity.com/paper/1607.07684/) (2016)
- [Regulation of Algorithmic Collusion](https://mlsanity.com/paper/2401.15794/) (2024)
- [No-Regret Algorithms in non-Truthful Auctions with Budget and ROI Constraints](https://mlsanity.com/paper/2404.09832/) (2024)
- [Correlated and Coarse equilibria of Single-item auctions](https://mlsanity.com/paper/1601.07702/) (2016)
- [Nash Convergence of Mean-Based Learning Algorithms in First-Price Auctions](https://mlsanity.com/paper/2110.03906/) (2021)
- [Strategically-Robust Learning Algorithms for Bidding in First-Price Auctions](https://mlsanity.com/paper/2402.07363/) (2024)

## Built on by (most cited first, 3 of 3)

- [From No-Regret to Strategically Robust Learning in Repeated Auctions](https://mlsanity.com/paper/2601.03853/) (2026)
- [Markets with Heterogeneous Agents: Dynamics and Survival of Bayesian vs. No-Regret Learners](https://mlsanity.com/paper/2502.08597/) (2025)
- [Equilibrium with Internal Transfers](https://mlsanity.com/paper/2606.20960/) (2026)

## Abstract (arXiv)

In repeated games, such as auctions, players rely on autonomous learning agents to choose their actions. We study settings in which players have their agents make monetary transfers to other agents during play at their own expense, in order to influence learning dynamics in their favor. Our goal is to understand when players have incentives to use such payments, how payments between agents affect learning outcomes, and what the resulting implications are for welfare and its distribution. We propose a simple game-theoretic model to capture the incentive structure of such scenarios. We find that, quite generally, abstaining from payments is not robust to strategic deviations by users of learning agents: self-interested players benefit from having their agents make payments to other learners. In a broad class of games, such endogenous payments between learning agents lead to higher welfare for all players. In first- and second-price auctions, equilibria of the induced "payment-policy game" lead to highly collusive learning outcomes, with low or vanishing revenue for the auctioneer. These results highlight a fundamental challenge for mechanism design, as well as for regulatory policies, in environments where learning agents may interact in the digital ecosystem beyond a mechanism's boundaries.