# Reinforcement Mechanism Design, with Applications to Dynamic Pricing in Sponsored Search Auctions

arXiv: https://arxiv.org/abs/1711.10279 · HTML: https://mlsanity.com/paper/1711.10279/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2017.

## Built on by (most cited first, 12 of 12)

- [A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising](https://mlsanity.com/paper/2106.06224/) (2021)
- [A Context-Integrated Transformer-Based Neural Network for Auction Design](https://mlsanity.com/paper/2201.12489/) (2022)
- [Learning to Mitigate AI Collusion on Economic Platforms](https://mlsanity.com/paper/2202.07106/) (2022)
- [Social World Model-Augmented Mechanism Design Policy Learning](https://mlsanity.com/paper/2510.19270/) (2025)
- [An Efficient Group-based Search Engine Marketing System for E-Commerce](https://mlsanity.com/paper/2106.12700/) (2021)
- [Carbon Market Simulation with Adaptive Mechanism Design](https://mlsanity.com/paper/2406.07875/) (2024)
- [A Survey of Online Auction Mechanism Design Using Deep Learning Approaches](https://mlsanity.com/paper/2110.06880/) (2021)
- [Deep Reinforcement Learning for Sequential Combinatorial Auctions](https://mlsanity.com/paper/2407.08022/) (2024)
- [Constant-Memory Strategies in Stochastic Games: Best Responses and Equilibria](https://mlsanity.com/paper/2505.07008/) (2025)
- [Mechanism Design for Ad Auctions with Display Prices](https://mlsanity.com/paper/2303.13192/) (2023)
- [Data on the Move: Traffic-Oriented Data Trading Platform Powered by AI Agent with Common Sense](https://mlsanity.com/paper/2407.00995/) (2024)
- [Universal Representation of Generalized Convex Functions and their Gradients](https://mlsanity.com/paper/2509.04477/) (2025)

## Abstract (arXiv)

In this study, we apply reinforcement learning techniques and propose what we call reinforcement mechanism design to tackle the dynamic pricing problem in sponsored search auctions. In contrast to previous game-theoretical approaches that heavily rely on rationality and common knowledge among the bidders, we take a data-driven approach, and try to learn, over repeated interactions, the set of optimal reserve prices. We implement our approach within the current sponsored search framework of a major search engine: we first train a buyer behavior model, via a real bidding data set, that accurately predicts bids given information that bidders are aware of, including the game parameters disclosed by the search engine, as well as the bidders' KPI data from previous rounds. We then put forward a reinforcement/MDP (Markov Decision Process) based algorithm that optimizes reserve prices over time, in a GSP-like auction. Our simulations demonstrate that our framework outperforms static optimization strategies including the ones that are currently in use, as well as several other dynamic ones.