# Multi-Head Online Learning for Delayed Feedback Modeling

arXiv: https://arxiv.org/abs/2205.12406 · HTML: https://mlsanity.com/paper/2205.12406/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 24 May 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Online Learning: A Comprehensive Survey](https://mlsanity.com/paper/1802.02871/) (2018)
- [Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction](https://mlsanity.com/paper/1907.06558/) (2019)
- [Field-aware Factorization Machines in a Real-world Online Advertising System](https://mlsanity.com/paper/1701.04099/) (2017)
- [Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modeling](https://mlsanity.com/paper/2104.14121/) (2021)
- [A Nonparametric Delayed Feedback Model for Conversion Rate Prediction](https://mlsanity.com/paper/1802.00255/) (2018)
- [Reacting to Variations in Product Demand: An Application for Conversion Rate (CR) Prediction in Sponsored Search](https://mlsanity.com/paper/1806.08211/) (2018)
- [Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling](https://mlsanity.com/paper/2012.03245/) (2020)
- [A Feedback Shift Correction in Predicting Conversion Rates under Delayed Feedback](https://mlsanity.com/paper/2002.02068/) (2020)
- [Display advertising: Estimating conversion probability efficiently](https://mlsanity.com/paper/1710.08583/) (2017)
- [Handling many conversions per click in modeling delayed feedback](https://mlsanity.com/paper/2101.02284/) (2021)
- [Dynamic Hierarchical Empirical Bayes: A Predictive Model Applied to Online Advertising](https://mlsanity.com/paper/1809.02213/) (2018)

## Abstract (arXiv)

In online advertising, it is highly important to predict the probability and the value of a conversion (e.g., a purchase). It not only impacts user experience by showing relevant ads, but also affects ROI of advertisers and revenue of marketplaces. Unlike clicks, which often occur within minutes after impressions, conversions are expected to happen over a long period of time (e.g., 30 days for online shopping). It creates a challenge, as the true labels are only available after the long delays. Either inaccurate labels (partial conversions) are used, or models are trained on stale data (e.g., from 30 days ago). The problem is more eminent in online learning, which focuses on the live performance on the latest data. In this paper, a novel solution is presented to address this challenge using multi-head modeling. Unlike traditional methods, it directly quantizes conversions into multiple windows, such as day 1, day 2, day 3-7, and day 8-30. A sub-model is trained specifically on conversions within each window. Label freshness is maximally preserved in early models (e.g., day 1 and day 2), while late conversions are accurately utilized in models with longer delays (e.g., day 8-30). It is shown to greatly exceed the performance of known methods in online learning experiments for both conversion rate (CVR) and value per click (VPC) predictions. Lastly, as a general method for delayed feedback modeling, it can be combined with any advanced ML techniques to further improve the performance.