# Semi-supervised Transfer Learning for Evaluation of Model Classification Performance

arXiv: https://arxiv.org/abs/2208.07927 · HTML: https://mlsanity.com/paper/2208.07927/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2022.

## Built on by (most cited first, 2 of 2)

- [Bridging Data Gaps in Healthcare: A Scoping Review of Transfer Learning in Biomedical Data Analysis](https://mlsanity.com/paper/2407.11034/) (2024)
- [SUDO: a framework for evaluating clinical artificial intelligence systems without ground-truth annotations](https://mlsanity.com/paper/2403.17011/) (2024)

## Abstract (arXiv)

In modern machine learning applications, frequent encounters of covariate shift and label scarcity have posed challenges to robust model training and evaluation. Numerous transfer learning methods have been developed to robustly adapt the model itself to some unlabeled target populations using existing labeled data in a source population. However, there is a paucity of literature on transferring performance metrics of a trained model. In this paper, we aim to evaluate the performance of a trained binary classifier on unlabeled target population based on receiver operating characteristic (ROC) analysis. We proposed bf Semi-supervised bf Transfer lbf Earning of bf Accuracy bf Measures (STEAM), an efficient three-step estimation procedure that employs 1) double-index modeling to construct calibrated density ratio weights and 2) robust imputation to leverage the large amount of unlabeled data to improve estimation efficiency. We establish the consistency and asymptotic normality of the proposed estimators under correct specification of either the density ratio model or the outcome model. We also correct for potential overfitting bias in the estimators in finite samples with cross-validation. We compare our proposed estimators to existing methods and show reductions in bias and gains in efficiency through simulations. We illustrate the practical utility of the proposed method on evaluating prediction performance of a phenotyping model for Rheumatoid Arthritis (RA) on a temporally evolving EHR cohort.