# Unsupervised Risk Estimation Using Only Conditional Independence Structure

arXiv: https://arxiv.org/abs/1606.05313 · HTML: https://mlsanity.com/paper/1606.05313/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2016.

## Built on by (most cited first, 17 of 17)

- [OpenGAN: Open-Set Recognition via Open Data Generation](https://mlsanity.com/paper/2104.02939/) (2021)
- [Assessing Generalization of SGD via Disagreement](https://mlsanity.com/paper/2106.13799/) (2021)
- [Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift](https://mlsanity.com/paper/2206.13089/) (2022)
- [Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles](https://mlsanity.com/paper/2106.15728/) (2021)
- [Predicting Out-of-Distribution Error with the Projection Norm](https://mlsanity.com/paper/2202.05834/) (2022)
- [Energy-based Automated Model Evaluation](https://mlsanity.com/paper/2401.12689/) (2024)
- [Label-Free Model Evaluation with Semi-Structured Dataset Representations](https://mlsanity.com/paper/2112.00694/) (2021)
- [Hybrid Open-set Segmentation with Synthetic Negative Data](https://mlsanity.com/paper/2301.08555/) (2023)
- [SURE: SUrvey REcipes for building reliable and robust deep networks](https://mlsanity.com/paper/2403.00543/) (2024)
- [Detecting OODs as datapoints with High Uncertainty](https://mlsanity.com/paper/2108.06380/) (2021)
- [DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing](https://mlsanity.com/paper/2111.11053/) (2021)
- [Evaluating multiple models using labeled and unlabeled data](https://mlsanity.com/paper/2501.11866/) (2025)
- [FUSE: Ensembling Verifiers with Zero Labeled Data](https://mlsanity.com/paper/2604.18547/) (2026)
- [Data Consistency for Weakly Supervised Learning](https://mlsanity.com/paper/2202.03987/) (2022)
- [Streaming algorithms for evaluating noisy judges on unlabeled data -- binary classification](https://mlsanity.com/paper/2306.01726/) (2023)
- [Probabilistic computation and uncertainty quantification with emerging covariance](https://mlsanity.com/paper/2305.19265/) (2023)
- [Mixture Proportion Estimation and Weakly-supervised Kernel Test for Conditional Independence](https://mlsanity.com/paper/2604.07191/) (2026)

## Abstract (arXiv)

We show how to estimate a model's test error from unlabeled data, on distributions very different from the training distribution, while assuming only that certain conditional independencies are preserved between train and test. We do not need to assume that the optimal predictor is the same between train and test, or that the true distribution lies in any parametric family. We can also efficiently differentiate the error estimate to perform unsupervised discriminative learning. Our technical tool is the method of moments, which allows us to exploit conditional independencies in the absence of a fully-specified model. Our framework encompasses a large family of losses including the log and exponential loss, and extends to structured output settings such as hidden Markov models.