# MetaSel: A Test Selection Approach for Fine-tuned DNN Models

arXiv: https://arxiv.org/abs/2503.17534 · HTML: https://mlsanity.com/paper/2503.17534/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 52 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems](https://mlsanity.com/paper/1803.07519/) (2018)
- [Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder](https://mlsanity.com/paper/2003.02977/) (2020)
- [MNIST-C: A Robustness Benchmark for Computer Vision](https://mlsanity.com/paper/1906.02337/) (2019)
- [Guiding Deep Learning System Testing using Surprise Adequacy](https://mlsanity.com/paper/1808.08444/) (2018)
- [Test Selection for Deep Learning Systems](https://mlsanity.com/paper/1904.13195/) (2019)
- [Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)](https://mlsanity.com/paper/2205.00664/) (2022)
- [Black-Box Testing of Deep Neural Networks Through Test Case Diversity](https://mlsanity.com/paper/2112.12591/) (2021)
- [Detecting Adversarial Examples and Other Misclassifications in Neural Networks by Introspection](https://mlsanity.com/paper/1905.09186/) (2019)
- [Revisiting Neuron Coverage Metrics and Quality of Deep Neural Networks](https://mlsanity.com/paper/2201.00191/) (2022)
- [DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks](https://mlsanity.com/paper/2303.04878/) (2023)
- [A Review and Refinement of Surprise Adequacy](https://mlsanity.com/paper/2103.05939/) (2021)
- [TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks](https://mlsanity.com/paper/2105.10113/) (2021)
- [A Uniform Framework for Anomaly Detection in Deep Neural Networks](https://mlsanity.com/paper/2110.03092/) (2021)
- [Evaluating the Robustness of Test Selection Methods for Deep Neural Networks](https://mlsanity.com/paper/2308.01314/) (2023)
- [TEASMA: A Practical Methodology for Test Adequacy Assessment of Deep Neural Networks](https://mlsanity.com/paper/2308.01311/) (2023)

## Built on by (most cited first, 3 of 3)

- [A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMs](https://mlsanity.com/paper/2601.08024/) (2026)
- [CAFD: Concept-Aware DNN Fault Detection using VLMs](https://mlsanity.com/paper/2605.24008/) (2026)
- [Test Case Prioritization for DNNs via Neural Collapse Instability](https://mlsanity.com/paper/2607.20046/) (2026)

## Abstract (arXiv)

Deep Neural Networks (DNNs) face challenges during deployment due to covariate shift, i.e., data distribution shifts between development and deployment contexts. Fine-tuning adapts pre-trained models to new contexts requiring smaller labeled sets. However, testing fine-tuned models under constrained labeling budgets remains a critical challenge. This paper introduces MetaSel, a new approach tailored for DNN models that have been fine-tuned to address covariate shift, to select tests from unlabeled inputs. MetaSel assumes that fine-tuned and pre-trained models share related data distributions and exhibit similar behaviors for many inputs. However, their behaviors diverge within the input subspace where fine-tuning alters decision boundaries, making those inputs more prone to misclassification. Unlike general approaches that rely solely on the DNN model and its input set, MetaSel leverages information from both the fine-tuned and pre-trained models and their behavioral differences to estimate misclassification probability for unlabeled test inputs, enabling more effective test selection. Our extensive empirical evaluation, comparing MetaSel against 11 state-of-the-art approaches and involving 68 fine-tuned models across weak, medium, and strong distribution shifts, demonstrates that MetaSel consistently delivers significant improvements in Test Relative Coverage (TRC) over existing baselines, particularly under highly constrained labeling budgets. MetaSel shows average TRC improvements of 28.46% to 56.18% over the most frequent second-best baselines while maintaining a high TRC median and low variability. Our results confirm MetaSel's practicality, robustness, and cost-effectiveness for test selection in the context of fine-tuned models.