# A Highly Efficient Diversity-based Input Selection for DNN Improvement Using VLMs

arXiv: https://arxiv.org/abs/2601.08024 · HTML: https://mlsanity.com/paper/2601.08024/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems](https://mlsanity.com/paper/1803.07519/) (2018)
- [Guiding Deep Learning System Testing using Surprise Adequacy](https://mlsanity.com/paper/1808.08444/) (2018)
- [Text-To-Concept (and Back) via Cross-Model Alignment](https://mlsanity.com/paper/2305.06386/) (2023)
- [A Survey on Deep Learning for Software Engineering](https://mlsanity.com/paper/2011.14597/) (2020)
- [Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)](https://mlsanity.com/paper/2205.00664/) (2022)
- [Boosting Operational DNN Testing Efficiency through Conditioning](https://mlsanity.com/paper/1906.02533/) (2019)
- [Black-Box Testing of Deep Neural Networks Through Test Case Diversity](https://mlsanity.com/paper/2112.12591/) (2021)
- [DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks](https://mlsanity.com/paper/2303.04878/) (2023)
- [Evaluating the Robustness of Test Selection Methods for Deep Neural Networks](https://mlsanity.com/paper/2308.01314/) (2023)
- [TEASMA: A Practical Methodology for Test Adequacy Assessment of Deep Neural Networks](https://mlsanity.com/paper/2308.01311/) (2023)
- [Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)](https://mlsanity.com/paper/2503.17416/) (2025)
- [MetaSel: A Test Selection Approach for Fine-tuned DNN Models](https://mlsanity.com/paper/2503.17534/) (2025)

## Built on by (most cited first, 1 of 1)

- [CAFD: Concept-Aware DNN Fault Detection using VLMs](https://mlsanity.com/paper/2605.24008/) (2026)

## Abstract (arXiv)

Maintaining or improving the performance of Deep Neural Networks (DNNs) through fine-tuning requires labeling newly collected inputs, a process that is often costly and time-consuming. To alleviate this problem, input selection approaches have been developed in recent years to identify small, yet highly informative subsets for labeling. Diversity-based selection is one of the most effective approaches for this purpose. However, they are often computationally intensive and lack scalability for large input sets, limiting their practical applicability. To address this challenge, we introduce Concept-Based Diversity (CBD), a novel and highly efficient diversity metric for image inputs that leverages Vision-Language Models (VLMs). Our results show that CBD exhibits a strong correlation with Geometric Diversity (GD), an established diversity metric, while requiring only a fraction of its computation time. Building on this finding, we propose a hybrid input selection approach that combines CBD with Margin, a simple uncertainty metric. We conduct a comprehensive evaluation across a diverse set of DNN models, input sets, selection budgets, and six most effective state-of-the-art selection baselines. The results demonstrate that the CBD-based selection consistently outperforms all baselines at guiding input selection to improve the DNN model. Furthermore, the CBD-based selection approach remains highly efficient, requiring selection times close to those of simple uncertainty-based methods such as Margin, even on larger input sets like ImageNet. These results confirm not only the effectiveness and computational advantage of the CBD-based approach, particularly compared to hybrid baselines, but also its scalability in repetitive and extensive input selection scenarios.