# Auto-bidding in real-time auctions via Oracle Imitation Learning (OIL)

arXiv: https://arxiv.org/abs/2412.11434 · HTML: https://mlsanity.com/paper/2412.11434/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 53 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Real-Time Bidding by Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1701.02490/) (2017)
- [Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1802.09756/) (2018)
- [Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1802.08365/) (2018)
- [Display Advertising with Real-Time Bidding (RTB) and Behavioural Targeting](https://mlsanity.com/paper/1610.03013/) (2016)
- [Deep Reinforcement Learning for Sponsored Search Real-time Bidding](https://mlsanity.com/paper/1803.00259/) (2018)
- [Real-time Bidding for Online Advertising: Measurement and Analysis](https://mlsanity.com/paper/1306.6542/) (2013)
- [AuctionNet: A Novel Benchmark for Decision-Making in Large-Scale Games](https://mlsanity.com/paper/2412.10798/) (2024)
- [Feedback Control of Real-Time Display Advertising](https://mlsanity.com/paper/1603.01055/) (2016)
- [A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising](https://mlsanity.com/paper/2106.06224/) (2021)
- [Bidding Machine: Learning to Bid for Directly Optimizing Profits in Display Advertising](https://mlsanity.com/paper/1803.02194/) (2018)
- [Attribution Modeling Increases Efficiency of Bidding in Display Advertising](https://mlsanity.com/paper/1707.06409/) (2017)
- [An Efficient Deep Distribution Network for Bid Shading in First-Price Auctions](https://mlsanity.com/paper/2107.06650/) (2021)
- [ROI-Constrained Bidding via Curriculum-Guided Bayesian Reinforcement Learning](https://mlsanity.com/paper/2206.05240/) (2022)
- [Bid Shading in The Brave New World of First-Price Auctions](https://mlsanity.com/paper/2009.01360/) (2020)
- [Lift-Based Bidding in Ad Selection](https://mlsanity.com/paper/1507.04811/) (2015)
- [LADDER: A Human-Level Bidding Agent for Large-Scale Real-Time Online Auctions](https://mlsanity.com/paper/1708.05565/) (2017)
- [MEBS: Multi-task End-to-end Bid Shading for Multi-slot Display Advertising](https://mlsanity.com/paper/2403.02607/) (2024)
- [A multi-criteria approach to approximate solution of multiple-choice knapsack problem](https://mlsanity.com/paper/1712.06723/) (2017)

## Built on by (most cited first, 3 of 3)

- [Arnold: A multi-task, multi-embodiment muscle transformer policy](https://mlsanity.com/paper/2508.18066/) (2025)
- [Multi-agent Auto-Bidding with Latent Graph Diffusion Models](https://mlsanity.com/paper/2503.05805/) (2025)
- [Large-Scale Auto-bidding with Nash Equilibrium Constraints](https://mlsanity.com/paper/2503.10304/) (2025)

## Abstract (arXiv)

Online advertising has become one of the most successful business models of the internet era. Impression opportunities are typically allocated through real-time auctions, where advertisers bid to secure advertisement slots. Deciding the best bid for an impression opportunity is challenging, due to the stochastic nature of user behavior and the variability of advertisement traffic over time. In this work, we propose a framework for training auto-bidding agents in multi-slot second-price auctions to maximize acquisitions (e.g., clicks, conversions) while adhering to budget and cost-per-acquisition (CPA) constraints. We exploit the insight that, after an advertisement campaign concludes, determining the optimal bids for each impression opportunity can be framed as a multiple-choice knapsack problem (MCKP) with a nonlinear objective. We propose an "oracle" algorithm that identifies a near-optimal combination of impression opportunities and advertisement slots, considering both past and future advertisement traffic data. This oracle solution serves as a training target for a student network which bids having access only to real-time information, a method we term Oracle Imitation Learning (OIL). Through numerical experiments, we demonstrate that OIL achieves superior performance compared to both online and offline reinforcement learning algorithms, offering improved sample efficiency. Notably, OIL shifts the complexity of training auto-bidding agents from crafting sophisticated learning algorithms to solving a nonlinear constrained optimization problem efficiently.