# Fine-Grained Dynamic Framework for Bias-Variance Joint Optimization on Data Missing Not at Random

arXiv: https://arxiv.org/abs/2405.15403 · HTML: https://mlsanity.com/paper/2405.15403/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 17 of 17)

- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://mlsanity.com/paper/1804.07931/) (2018)
- [AutoDebias: Learning to Debias for Recommendation](https://mlsanity.com/paper/2105.04170/) (2021)
- [Large-scale Causal Approaches to Debiasing Post-click Conversion Rate Estimation with Multi-task Learning](https://mlsanity.com/paper/1910.09337/) (2019)
- [Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising](https://mlsanity.com/paper/2105.08489/) (2021)
- [Information Theoretic Counterfactual Learning from Missing-Not-At-Random Feedback](https://mlsanity.com/paper/2009.02623/) (2020)
- [Missing Not at Random in Matrix Completion: The Effectiveness of Estimating Missingness Probabilities Under a Low Nuclear Norm Assumption](https://mlsanity.com/paper/1910.12774/) (2019)
- [Asymmetric Tri-training for Debiasing Missing-Not-At-Random Explicit Feedback](https://mlsanity.com/paper/1910.01444/) (2019)
- [Enhanced Doubly Robust Learning for Debiasing Post-click Conversion Rate Estimation](https://mlsanity.com/paper/2105.13623/) (2021)
- [Rethinking Cross-Domain Sequential Recommendation under Open-World Assumptions](https://mlsanity.com/paper/2311.04590/) (2023)
- [TDR-CL: Targeted Doubly Robust Collaborative Learning for Debiased Recommendations](https://mlsanity.com/paper/2203.10258/) (2022)
- [A Generalized Doubly Robust Learning Framework for Debiasing Post-Click Conversion Rate Prediction](https://mlsanity.com/paper/2211.06684/) (2022)
- [Multiple Robust Learning for Recommendation](https://mlsanity.com/paper/2207.10796/) (2022)
- [Neural Node Matching for Multi-Target Cross Domain Recommendation](https://mlsanity.com/paper/2302.05919/) (2023)
- [Doubly Calibrated Estimator for Recommendation on Data Missing Not At Random](https://mlsanity.com/paper/2403.00817/) (2024)
- [Debiased Collaborative Filtering with Kernel-Based Causal Balancing](https://mlsanity.com/paper/2404.19596/) (2024)
- [Towards Open-world Cross-Domain Sequential Recommendation: A Model-Agnostic Contrastive Denoising Approach](https://mlsanity.com/paper/2311.04760/) (2023)

## Built on by (most cited first, 4 of 4)

- [Dynamic Momentum Recalibration in Online Gradient Learning](https://mlsanity.com/paper/2603.06120/) (2026)
- [Causal Label Recovery in Payment Networks](https://mlsanity.com/paper/2605.29272/) (2026)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [Addressing Correlated Latent Exogenous Variables in Debiased Recommender Systems](https://mlsanity.com/paper/2506.07517/) (2025)

## Abstract (arXiv)

In most practical applications such as recommendation systems, display advertising, and so forth, the collected data often contains missing values and those missing values are generally missing-not-at-random, which deteriorates the prediction performance of models. Some existing estimators and regularizers attempt to achieve unbiased estimation to improve the predictive performance. However, variances and generalization bound of these methods are generally unbounded when the propensity scores tend to zero, compromising their stability and robustness. In this paper, we first theoretically reveal that limitations of regularization techniques. Besides, we further illustrate that, for more general estimators, unbiasedness will inevitably lead to unbounded variance. These general laws inspire us that the estimator designs is not merely about eliminating bias, reducing variance, or simply achieve a bias-variance trade-off. Instead, it involves a quantitative joint optimization of bias and variance. Then, we develop a systematic fine-grained dynamic learning framework to jointly optimize bias and variance, which adaptively selects an appropriate estimator for each user-item pair according to the predefined objective function. With this operation, the generalization bounds and variances of models are reduced and bounded with theoretical guarantees. Extensive experiments are conducted to verify the theoretical results and the effectiveness of the proposed dynamic learning framework.