# Inferential Evaluation of Surrogate-Derived Models under Covariate Shift

arXiv: https://arxiv.org/abs/2608.15783 · HTML: https://mlsanity.com/paper/2608.15783/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 9 of 9)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [Automatic Debiased Machine Learning for Covariate Shifts](https://mlsanity.com/paper/2307.04527/) (2023)
- [Doubly Robust Augmented Model Accuracy Transfer Inference with High Dimensional Features](https://mlsanity.com/paper/2208.05134/) (2022)
- [Semi-Supervised Approaches to Efficient Evaluation of Model Prediction Performance](https://mlsanity.com/paper/1711.05663/) (2017)
- [Surrogate Assisted Semi-supervised Inference for High Dimensional Risk Prediction](https://mlsanity.com/paper/2105.01264/) (2021)
- [Efficient and Multiply Robust Risk Estimation under General Forms of Dataset Shift](https://mlsanity.com/paper/2306.16406/) (2023)
- [Towards the Efficient Inference by Incorporating Automated Computational Phenotypes under Covariate Shift](https://mlsanity.com/paper/2505.22632/) (2025)
- [Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias](https://mlsanity.com/paper/2606.14506/) (2026)

## Abstract (arXiv)

In transfer-learning settings, a model derived from abundant surrogate labels may be deployed in a target population where gold-standard outcomes are unobserved. Evaluating its target performance is essential for determining whether decisions based on the model remain reliable, yet it is difficult when gold labels are scarce, and covariate distributions differ across data sources. We study a three-sample setting with a small gold-labeled source, a larger surrogate-labeled source, and an unlabeled target. Under conditional transportability, we evaluate the surrogate-derived model against the latent gold-standard outcome in the target population. We propose cross-fitted estimators that transport information from the two labeled sources through source-specific density ratios. We also combine outcome-regression augmentation with a kernel correction for estimating the model near a threshold, accounting for uncertainty from all three samples. We establish asymptotically linear inference for TPR and FPR, consistency and pointwise inference for the ROC curve, and asymptotically normal inference for AUC. Simulations assess bias, coverage, and sensitivity to bandwidth and relative sample sizes. A retrospective temporal validation on Chatbot Arena and a semi-synthetic ACS-Income study provide validation in real-world AI applications.