# A Unified Framework for Inference with General Missingness Patterns and Machine Learning Imputation

arXiv: https://arxiv.org/abs/2508.15162 · HTML: https://mlsanity.com/paper/2508.15162/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 16 of 16)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [PPI++: Efficient Prediction-Powered Inference](https://mlsanity.com/paper/2311.01453/) (2023)
- [Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Models](https://mlsanity.com/paper/2306.04746/) (2023)
- [Predictions as Surrogates: Revisiting Surrogate Outcomes in the Age of AI](https://mlsanity.com/paper/2501.09731/) (2025)
- [Assumption-Lean and Data-Adaptive Post-Prediction Inference](https://mlsanity.com/paper/2311.14220/) (2023)
- [Prediction-Powered Inference with Imputed Covariates and Nonuniform Sampling](https://mlsanity.com/paper/2501.18577/) (2025)
- [Prediction-powered Generalization of Causal Inferences](https://mlsanity.com/paper/2406.02873/) (2024)
- [Multiple Imputation: A Review of Practical and Theoretical Findings](https://mlsanity.com/paper/1801.04058/) (2018)
- [Prediction De-Correlated Inference: A safe approach for post-prediction inference](https://mlsanity.com/paper/2312.06478/) (2023)
- [A Unified Framework for Semiparametrically Efficient Semi-Supervised Learning](https://mlsanity.com/paper/2502.17741/) (2025)
- [Bayesian Prediction-Powered Inference](https://mlsanity.com/paper/2405.06034/) (2024)
- [AI-Powered Bayesian Inference](https://mlsanity.com/paper/2502.19231/) (2025)
- [Another look at statistical inference with machine learning-imputed data](https://mlsanity.com/paper/2411.19908/) (2024)
- [On Inverse Probability Weighting for Nonmonotone Missing at Random Data](https://mlsanity.com/paper/1411.5310/) (2014)
- [ipd: An R Package for Conducting Inference on Predicted Data](https://mlsanity.com/paper/2410.09665/) (2024)

## Built on by (most cited first, 7 of 7)

- [Demystifying Prediction Powered Inference](https://mlsanity.com/paper/2601.20819/) (2026)
- [Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Data](https://mlsanity.com/paper/2512.05456/) (2025)
- [Spatially Robust Inference with Predicted and Missing at Random Labels](https://mlsanity.com/paper/2603.11368/) (2026)
- [Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias](https://mlsanity.com/paper/2606.14506/) (2026)
- [When Surveys Become Conversations: Adaptive Matrix Validation for AI-Assisted Interviews](https://mlsanity.com/paper/2606.24244/) (2026)
- [ABC-Align: Prediction-Powered Alignment with Adaptive Bias Control](https://mlsanity.com/paper/2609.34374/) (2026)
- [Assumption-lean logistic regression with missing covariates](https://mlsanity.com/paper/2610.07292/) (2026)

## Abstract (arXiv)

Pre-trained machine learning (ML) predictions have been increasingly used to complement incomplete data to enable downstream scientific inquiries, but their naive integration risks biased inferences. Recently, multiple methods have been developed to provide valid inference with ML imputations regardless of prediction quality and to enhance efficiency relative to complete-case analyses. However, existing approaches are often limited to missing outcomes under a missing-completely-at-random (MCAR) assumption, failing to handle general missingness patterns (missing in both the outcome and exposures) under the more realistic missing-at-random (MAR) assumption. This paper develops a novel method that delivers a valid statistical inference framework for general Z-estimation problems using ML imputations under the MAR assumption and for general missingness patterns. The core technical idea is to stratify observations by distinct missingness patterns and construct an estimator by appropriately weighting and aggregating pattern-specific information through a masking-and-imputation procedure on the complete cases. We provide theoretical guarantees of asymptotic normality of the proposed estimator and efficiency dominance over weighted complete-case analyses. Practically, the method affords simple implementations by leveraging existing weighted complete-case analysis software. Extensive simulations are carried out to validate theoretical results. A real data example is provided to further illustrate the practical utility of the proposed method. The paper concludes with a brief discussion on practical implications, limitations, and potential future directions.