# Functional Optimization Reinforcement Learning for Real-Time Bidding

arXiv: https://arxiv.org/abs/2206.13939 · HTML: https://mlsanity.com/paper/2206.13939/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 25 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Real-Time Bidding by Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1701.02490/) (2017)
- [Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1802.09756/) (2018)
- [Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1802.08365/) (2018)
- [Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising](https://mlsanity.com/paper/1806.03514/) (2018)
- [Optimized Cost per Click in Taobao Display Advertising](https://mlsanity.com/paper/1703.02091/) (2017)
- [Display Advertising with Real-Time Bidding (RTB) and Behavioural Targeting](https://mlsanity.com/paper/1610.03013/) (2016)
- [Bid Optimization by Multivariable Control in Display Advertising](https://mlsanity.com/paper/1905.10928/) (2019)
- [Bidding Machine: Learning to Bid for Directly Optimizing Profits in Display Advertising](https://mlsanity.com/paper/1803.02194/) (2018)
- [Smart Pacing for Effective Online Ad Campaign Optimization](https://mlsanity.com/paper/1506.05851/) (2015)
- [Real Time Bid Optimization with Smooth Budget Delivery in Online Advertising](https://mlsanity.com/paper/1305.3011/) (2013)
- [Profit Maximization for Online Advertising Demand-Side Platforms](https://mlsanity.com/paper/1706.01614/) (2017)
- [Learning Utilities and Equilibria in Non-Truthful Auctions](https://mlsanity.com/paper/2007.01722/) (2020)
- [Field-Embedded Factorization Machines for Click-through rate prediction](https://mlsanity.com/paper/2009.09931/) (2020)

## Abstract (arXiv)

Real-time bidding is the new paradigm of programmatic advertising. An advertiser wants to make the intelligent choice of utilizing a Demand-Side Platform to improve the performance of their ad campaigns. Existing approaches are struggling to provide a satisfactory solution for bidding optimization due to stochastic bidding behavior. In this paper, we proposed a multi-agent reinforcement learning architecture for RTB with functional optimization. We designed four agents bidding environment: three Lagrange-multiplier based functional optimization agents and one baseline agent (without any attribute of functional optimization) First, numerous attributes have been assigned to each agent, including biased or unbiased win probability, Lagrange multiplier, and click-through rate. In order to evaluate the proposed RTB strategy's performance, we demonstrate the results on ten sequential simulated auction campaigns. The results show that agents with functional actions and rewards had the most significant average winning rate and winning surplus, given biased and unbiased winning information respectively. The experimental evaluations show that our approach significantly improve the campaign's efficacy and profitability.