# Improving Real-Time Bidding in Online Advertising Using Markov Decision Processes and Machine Learning Techniques

arXiv: https://arxiv.org/abs/2305.04889 · HTML: https://mlsanity.com/paper/2305.04889/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 23 references that are in the map.
- First posted on arXiv 5 May 2023.

## Builds on (most cited first, 6 of 6)

- [Real-Time Bidding by Reinforcement Learning in Display Advertising](https://mlsanity.com/paper/1701.02490/) (2017)
- [Stochastic Bandits for Multi-platform Budget Optimization in Online Advertising](https://mlsanity.com/paper/2103.10246/) (2021)
- [Real-Time Bidding Benchmarking with iPinYou Dataset](https://mlsanity.com/paper/1407.7073/) (2014)
- [Deep Landscape Forecasting for Real-time Bidding Advertising](https://mlsanity.com/paper/1905.03028/) (2019)
- [Scalable Bid Landscape Forecasting in Real-time Bidding](https://mlsanity.com/paper/2001.06587/) (2020)
- [Online Advertising Revenue Forecasting: An Interpretable Deep Learning Approach](https://mlsanity.com/paper/2111.08840/) (2021)

## Abstract (arXiv)

Real-time bidding has emerged as an effective online advertising technique. With real-time bidding, advertisers can position ads per impression, enabling them to optimise ad campaigns by targeting specific audiences in real-time. This paper proposes a novel method for real-time bidding that combines deep learning and reinforcement learning techniques to enhance the efficiency and precision of the bidding process. In particular, the proposed method employs a deep neural network to predict auction details and market prices and a reinforcement learning algorithm to determine the optimal bid price. The model is trained using historical data from the iPinYou dataset and compared to cutting-edge real-time bidding algorithms. The outcomes demonstrate that the proposed method is preferable regarding cost-effectiveness and precision. In addition, the study investigates the influence of various model parameters on the performance of the proposed algorithm. It offers insights into the efficacy of the combined deep learning and reinforcement learning approach for real-time bidding. This study contributes to advancing techniques and offers a promising direction for future research.