# Semiparametric Learning from Open-Set Label Shift Data

arXiv: https://arxiv.org/abs/2509.14522 · HTML: https://mlsanity.com/paper/2509.14522/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 14 of 14)

- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Universal Domain Adaptation through Self Supervision](https://mlsanity.com/paper/2002.07953/) (2020)
- [Mixture Proportion Estimation via Kernel Embedding of Distributions](https://mlsanity.com/paper/1603.02501/) (2016)
- [Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels](https://mlsanity.com/paper/1705.01936/) (2017)
- [Mixture Proportion Estimation and PU Learning: A Modern Approach](https://mlsanity.com/paper/2111.00980/) (2021)
- [Prediction and outlier detection in classification problems](https://mlsanity.com/paper/1905.04396/) (2019)
- [Domain Adaptation under Open Set Label Shift](https://mlsanity.com/paper/2207.13048/) (2022)
- [Estimating the class prior and posterior from noisy positives and unlabeled data](https://mlsanity.com/paper/1606.08561/) (2016)
- [Mixture Proportion Estimation Beyond Irreducibility](https://mlsanity.com/paper/2306.01253/) (2023)
- [Class Proportion Estimation with Application to Multiclass Anomaly Rejection](https://mlsanity.com/paper/1306.5056/) (2013)
- [Neyman-Pearson Multi-class Classification via Cost-sensitive Learning](https://mlsanity.com/paper/2111.04597/) (2021)
- [Positive and Unlabeled Data: Model, Estimation, Inference, and Classification](https://mlsanity.com/paper/2407.09735/) (2024)
- [PULasso: High-dimensional variable selection with presence-only data](https://mlsanity.com/paper/1711.08129/) (2017)
- [High-dimensional Multi-class Classification with Presence-only Data](https://mlsanity.com/paper/2304.09305/) (2023)

## Abstract (arXiv)

We study the open-set label shift problem, where the test data may include a novel class absent from training. This setting is challenging because both the class proportions and the distribution of the novel class are not identifiable without extra assumptions. Existing approaches often rely on restrictive separability conditions, prior knowledge, or computationally infeasible procedures, and some may lack theoretical guarantees. We propose a semiparametric density ratio model framework that ensures identifiability while allowing overlap between novel and known classes. Within this framework, we develop maximum empirical likelihood estimators and confidence intervals for class proportions, establish their asymptotic validity, and design a stable Expectation-Maximization algorithm for computation. We further construct an approximately optimal classifier based on posterior probabilities with theoretical guarantees. Simulations and a real data application confirm that our methods improve both estimation accuracy and classification performance compared with existing approaches.