Data to 14 Oct 2026

Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation

arXiv 2202.06881 ↗First posted 14 Feb 2022cs.LG stat.MLcited by 26

Not on the map. Why?

Abstract (arXiv)

We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation. Evaluating model performance is a challenging and important problem when labels are expensive. ASEs address this active testing problem using a surrogate-based estimation approach that interpolates the errors of points with unknown labels, rather than forming a Monte Carlo estimator. ASEs actively learn the underlying surrogate, and we propose a novel acquisition strategy, XWED, that tailors this learning to the final estimation task. We find that ASEs offer greater label-efficiency than the current state-of-the-art when applied to challenging model evaluation problems for deep neural networks.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 32 of its 82 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

32 of 82 references.

References · 32

2015Deep Residual Learning for Image Recognitioncited by 52,096
2014Adam: A Method for Stochastic Optimizationcited by 37,240
2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
2016Wide Residual Networkscited by 3,070
2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
2017Deep Bayesian Active Learning with Image Datacited by 819
2019AutoML: A Survey of the State-of-the-Artcited by 390
2018Meta-Learning: A Surveycited by 256
2017Advances in Variational Inferencecited by 236

20 more

2020Hyper-Parameter Optimization: A Review of Algorithms and Applicationscited by 114
2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 106
2020Are Labels Always Necessary for Classifier Accuracy Evaluation?cited by 104
2021Active Testing: Sample-Efficient Model Evaluationcited by 82
2021On Statistical Bias In Active Learning: How and When To Fix Itcited by 75
2018Practical Obstacles to Deploying Active Learningcited by 68
2021What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?cited by 63
2020Computing the Testing Error without a Testing Setcited by 54
2019Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learningcited by 37
2015Probabilistic Integration: A Role in Statistical Computation?cited by 26
2015Convergence Rates of Active Learning for Maximum Likelihood Estimationcited by 25
2019Convergence Guarantees for Adaptive Bayesian Quadrature Methodscited by 22
2020Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?cited by 21
2021Convergence of Uncertainty Sampling for Active Learningcited by 19
2020Active Bayesian Assessment for Black-Box Classifierscited by 17
2021Targeted Active Learning for Bayesian Decision-Makingcited by 12
2021Label-Free Model Evaluation with Semi-Structured Dataset Representationscited by 10
2014Active Regression by Stratificationcited by 8
2016Classifier Risk Estimation under Limited Labeling Resourcescited by 7
2021Sample Efficient Model Evaluationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 7; 2026: 10.

26 papers.

Sort:

By area:

2023Modern Bayesian Experimental Designcited by 125
2023Anchor Points: Benchmarking Models with Much Fewer Examplescited by 86
2025How Benchmark Prediction from Fewer Data Misses the Markcited by 28
2024AcTracer: Active Testing of Large Language Model via Multi-Stage Samplingcited by 17
2024A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimationcited by 14
2025Scaling Up Active Testing to Large Language Modelscited by 13
2024FlashEval: Towards Fast and Accurate Evaluation of Text-to-image Diffusion Generative Modelscited by 11
2024Efficient Lifelong Model Evaluation in an Era of Rapid Progresscited by 10
2025Consensus-Driven Active Model Selectioncited by 8
2025DISCO: Diversifying Sample Condensation for Efficient Model Evaluationcited by 5
2026Active Testing of Large Language Models via Approximate Neyman Allocationcited by 5
2025Evaluating multiple models using labeled and unlabeled datacited by 4

14 more

2023Learning to Rank for Active Learning via Multi-Task Bilevel Optimizationcited by 3
2024Exploring Human-in-the-Loop Test-Time Adaptation by Synergizing Active Learning and Model Selectioncited by 2
2026Interpreting and Controlling Model Behavior via Constitutions for Atomic Concept Editscited by 2
2026Generative Active Testing: Efficient LLM Evaluation via Proxy Task Adaptationcited by 2
2026ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluationcited by 2
2025Incremental Uncertainty-aware Performance Monitoring with Active Labeling Interventioncited by 1
2026DeltaSelect: Affordable A/B Testing for Coding Agentscited by 1
2024Novel Deep Neural Network Classifier Characterization Metrics with Applications to Dataless Evaluationcited by 0
2025Using Synthetic Data to estimate the True Error is theoretically and practically doablecited by 0
2026Label-Efficient Monitoring of Classification Models via Stratified Importance Samplingcited by 0
2026ShaplEIG: Bayesian Experimental Design for Shapley Value Estimationcited by 0
2026MLIP Detective: Active Failure Mode Discovery Beyond Benchmark Scores for Machine-Learning Interatomic Potentialscited by 0
2026Targeted Review for AI-Assisted Biodiversity Surveys: Active Continuous-Score Occupancy Modelingcited by 0
2026Speculative Evaluation of Stochastic LLMscited by 0