# Sample Efficient Model Evaluation

arXiv: https://arxiv.org/abs/2109.12043 · HTML: https://mlsanity.com/paper/2109.12043/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 17 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 2 of 2)

- [Active Testing: Sample-Efficient Model Evaluation](https://mlsanity.com/paper/2103.05331/) (2021)
- [Active Testing: An Efficient and Robust Framework for Estimating Accuracy](https://mlsanity.com/paper/1807.00493/) (2018)

## Built on by (most cited first, 4 of 4)

- [Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation](https://mlsanity.com/paper/2202.06881/) (2022)
- [Scaling Up Active Testing to Large Language Models](https://mlsanity.com/paper/2508.09093/) (2025)
- [Efficient Evaluation of Multi-Task Robot Policies With Active Experiment Selection](https://mlsanity.com/paper/2502.09829/) (2025)
- [Prediction-Powered Active Testing](https://mlsanity.com/paper/2607.08347/) (2026)

## Abstract (arXiv)

Labelling data is a major practical bottleneck in training and testing classifiers. Given a collection of unlabelled data points, we address how to select which subset to label to best estimate test metrics such as accuracy, F1 score or micro/macro F1. We consider two sampling based approaches, namely the well-known Importance Sampling and we introduce a novel application of Poisson Sampling. For both approaches we derive the minimal error sampling distributions and how to approximate and use them to form estimators and confidence intervals. We show that Poisson Sampling outperforms Importance Sampling both theoretically and experimentally.