# A Feedback Shift Correction in Predicting Conversion Rates under Delayed Feedback

arXiv: https://arxiv.org/abs/2002.02068 · HTML: https://mlsanity.com/paper/2002.02068/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2020.

## Built on by (most cited first, 17 of 17)

- [Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modeling](https://mlsanity.com/paper/2104.14121/) (2021)
- [Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction](https://mlsanity.com/paper/2202.06472/) (2022)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [Ads Recommendation in a Collapsed and Entangled World](https://mlsanity.com/paper/2403.00793/) (2024)
- [Generalized Delayed Feedback Model with Post-Click Information in Recommender Systems](https://mlsanity.com/paper/2206.00407/) (2022)
- [Follow the Prophet: Accurate Online Conversion Rate Prediction in the Face of Delayed Feedback](https://mlsanity.com/paper/2108.06167/) (2021)
- [Handling many conversions per click in modeling delayed feedback](https://mlsanity.com/paper/2101.02284/) (2021)
- [Modeling Cascaded Delay Feedback for Online Net Conversion Rate Prediction: Benchmark, Insights and Solutions](https://mlsanity.com/paper/2601.19965/) (2026)
- [Personalized Interpolation: Achieving Efficient Conversion Estimation with Flexible Optimization Windows](https://mlsanity.com/paper/2501.14103/) (2025)
- [Delayed Feedback Modeling with Influence Functions](https://mlsanity.com/paper/2502.01669/) (2025)
- [Multi-Head Online Learning for Delayed Feedback Modeling](https://mlsanity.com/paper/2205.12406/) (2022)
- [Freshness or Accuracy, Why Not Both? Addressing Delayed Feedback via Dynamic Graph Neural Networks](https://mlsanity.com/paper/2308.08071/) (2023)
- [Conversion rate prediction in online advertising: modeling techniques, performance evaluation and future directions](https://mlsanity.com/paper/2512.01171/) (2025)
- [Linear Convergence in Games with Delayed Feedback via Extra Prediction](https://mlsanity.com/paper/2602.17486/) (2026)
- [Follow the TRACE: Exploiting Post-Click Trajectories for Online Delayed Conversion Rate Prediction](https://mlsanity.com/paper/2604.23197/) (2026)
- [TWICE: Two-Clock, Two-Window Learning for Long-Horizon Conversion Prediction in Online Advertising](https://mlsanity.com/paper/2607.25404/) (2026)
- [DMRL: Document-Mediated Reinforcement Learning for Skill Optimization in Advertising Recommendation](https://mlsanity.com/paper/2609.02170/) (2026)

## Abstract (arXiv)

In display advertising, predicting the conversion rate, that is, the probability that a user takes a predefined action on an advertiser's website, such as purchasing goods is fundamental in estimating the value of displaying the advertisement. However, there is a relatively long time delay between a click and its resultant conversion. Because of the delayed feedback, some positive instances at the training period are labeled as negative because some conversions have not yet occurred when training data are gathered. As a result, the conditional label distributions differ between the training data and the production environment. This situation is referred to as a feedback shift. We address this problem by using an importance weight approach typically used for covariate shift correction. We prove its consistency for the feedback shift. Results in both offline and online experiments show that our proposed method outperforms the existing method.