# Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction

arXiv: https://arxiv.org/abs/2202.06472 · HTML: https://mlsanity.com/paper/2202.06472/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 14 Feb 2022.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Online Learning under Delayed Feedback](https://mlsanity.com/paper/1306.0686/) (2013)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)
- [Stochastic Bandit Models for Delayed Conversions](https://mlsanity.com/paper/1706.09186/) (2017)
- [Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction](https://mlsanity.com/paper/1907.06558/) (2019)
- [Predicting Different Types of Conversions with Multi-Task Learning in Online Advertising](https://mlsanity.com/paper/1907.10235/) (2019)
- [Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modeling](https://mlsanity.com/paper/2104.14121/) (2021)
- [A Nonparametric Delayed Feedback Model for Conversion Rate Prediction](https://mlsanity.com/paper/1802.00255/) (2018)
- [Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling](https://mlsanity.com/paper/2012.03245/) (2020)
- [A Feedback Shift Correction in Predicting Conversion Rates under Delayed Feedback](https://mlsanity.com/paper/2002.02068/) (2020)
- [Modeling Perception Errors towards Robust Decision Making in Autonomous Vehicles](https://mlsanity.com/paper/2001.11695/) (2020)
- [Gradient-free Online Learning in Games with Delayed Rewards](https://mlsanity.com/paper/2006.10911/) (2020)
- [Delayed Feedback Modeling for the Entire Space Conversion Rate Prediction](https://mlsanity.com/paper/2011.11826/) (2020)
- [PBODL : Parallel Bayesian Online Deep Learning for Click-Through Rate Prediction in Tencent Advertising System](https://mlsanity.com/paper/1707.00802/) (2017)
- [Adversarial Mixture Of Experts with Category Hierarchy Soft Constraint](https://mlsanity.com/paper/2007.12349/) (2020)

## Built on by (most cited first, 15 of 15)

- [Generalized Delayed Feedback Model with Post-Click Information in Recommender Systems](https://mlsanity.com/paper/2206.00407/) (2022)
- [Capturing Conversion Rate Fluctuation during Sales Promotions: A Novel Historical Data Reuse Approach](https://mlsanity.com/paper/2305.12837/) (2023)
- [See Beyond a Single View: Multi-Attribution Learning Leads to Better Conversion Rate Prediction](https://mlsanity.com/paper/2508.15217/) (2025)
- [Ensure Timeliness and Accuracy: A Novel Sliding Window Data Stream Paradigm for Live Streaming Recommendation](https://mlsanity.com/paper/2402.14399/) (2024)
- [Modeling Cascaded Delay Feedback for Online Net Conversion Rate Prediction: Benchmark, Insights and Solutions](https://mlsanity.com/paper/2601.19965/) (2026)
- [Delayed Feedback Modeling for Post-Click Gross Merchandise Volume Prediction: Benchmark, Insights and Approaches](https://mlsanity.com/paper/2601.20307/) (2026)
- [Personalized Interpolation: Achieving Efficient Conversion Estimation with Flexible Optimization Windows](https://mlsanity.com/paper/2501.14103/) (2025)
- [Delayed Feedback Modeling with Influence Functions](https://mlsanity.com/paper/2502.01669/) (2025)
- [MAC: A Conversion Rate Prediction Benchmark Featuring Labels Under Multiple Attribution Mechanisms](https://mlsanity.com/paper/2603.02184/) (2026)
- [Freshness or Accuracy, Why Not Both? Addressing Delayed Feedback via Dynamic Graph Neural Networks](https://mlsanity.com/paper/2308.08071/) (2023)
- [Conversion rate prediction in online advertising: modeling techniques, performance evaluation and future directions](https://mlsanity.com/paper/2512.01171/) (2025)
- [Statistical Learning from Attribution Sets](https://mlsanity.com/paper/2602.06276/) (2026)
- [Follow the TRACE: Exploiting Post-Click Trajectories for Online Delayed Conversion Rate Prediction](https://mlsanity.com/paper/2604.23197/) (2026)
- [TWICE: Two-Clock, Two-Window Learning for Long-Horizon Conversion Prediction in Online Advertising](https://mlsanity.com/paper/2607.25404/) (2026)
- [DMRL: Document-Mediated Reinforcement Learning for Skill Optimization in Advertising Recommendation](https://mlsanity.com/paper/2609.02170/) (2026)

## Abstract (arXiv)

Alleviating the delayed feedback problem is of crucial importance for the conversion rate(CVR) prediction in online advertising. Previous delayed feedback modeling methods using an observation window to balance the trade-off between waiting for accurate labels and consuming fresh feedback. Moreover, to estimate CVR upon the freshly observed but biased distribution with fake negatives, the importance sampling is widely used to reduce the distribution bias. While effective, we argue that previous approaches falsely treat fake negative samples as real negative during the importance weighting and have not fully utilized the observed positive samples, leading to suboptimal performance. In this work, we propose a new method, DElayed Feedback modeling with UnbiaSed Estimation, (DEFUSE), which aim to respectively correct the importance weights of the immediate positive, the fake negative, the real negative, and the delay positive samples at finer granularity. Specifically, we propose a two-step optimization approach that first infers the probability of fake negatives among observed negatives before applying importance sampling. To fully exploit the ground-truth immediate positives from the observed distribution, we further develop a bi-distribution modeling framework to jointly model the unbiased immediate positives and the biased delay conversions. Experimental results on both public and our industrial datasets validate the superiority of DEFUSE. Codes are available at https://github.com/ychen216/DEFUSE.git.