# Cost-Effective Training of Deep CNNs with Active Model Adaptation

arXiv: https://arxiv.org/abs/1802.05394 · HTML: https://mlsanity.com/paper/1802.05394/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2018.

## Built on by (most cited first, 16 of 16)

- [On Statistical Bias In Active Learning: How and When To Fix It](https://mlsanity.com/paper/2101.11665/) (2021)
- [Active Learning for Open-set Annotation](https://mlsanity.com/paper/2201.06758/) (2022)
- [Multi-Anchor Active Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2108.08012/) (2021)
- [Dirichlet-based Uncertainty Calibration for Active Domain Adaptation](https://mlsanity.com/paper/2302.13824/) (2023)
- [Learning Distinctive Margin toward Active Domain Adaptation](https://mlsanity.com/paper/2203.05738/) (2022)
- [Divide and Adapt: Active Domain Adaptation via Customized Learning](https://mlsanity.com/paper/2307.11618/) (2023)
- [SALAD: Source-free Active Label-Agnostic Domain Adaptation for Classification, Segmentation and Detection](https://mlsanity.com/paper/2205.12840/) (2022)
- [MADAv2: Advanced Multi-Anchor Based Active Domain Adaptation Segmentation](https://mlsanity.com/paper/2301.07354/) (2023)
- [Loss-based Sequential Learning for Active Domain Adaptation](https://mlsanity.com/paper/2204.11665/) (2022)
- [Understanding Approximation for Bayesian Inference in Neural Networks](https://mlsanity.com/paper/2211.06139/) (2022)
- [Convex Online Video Frame Subset Selection using Multiple Criteria for Data Efficient Autonomous Driving](https://mlsanity.com/paper/2103.13021/) (2021)
- [Deep Active Learning with Budget Annotation](https://mlsanity.com/paper/2208.00508/) (2022)
- [Dynamic Domain Discrepancy Adjustment for Active Multi-Domain Adaptation](https://mlsanity.com/paper/2307.14068/) (2023)
- [VTruST: Controllable value function based subset selection for Data-Centric Trustworthy AI](https://mlsanity.com/paper/2403.05174/) (2024)
- [Salutary Labeling with Zero Human Annotation](https://mlsanity.com/paper/2405.17627/) (2024)
- [PEAKS: Selecting Key Training Examples Incrementally via Prediction Error Anchored by Kernel Similarity](https://mlsanity.com/paper/2504.05250/) (2025)

## Abstract (arXiv)

Deep convolutional neural networks have achieved great success in various applications. However, training an effective DNN model for a specific task is rather challenging because it requires a prior knowledge or experience to design the network architecture, repeated trial-and-error process to tune the parameters, and a large set of labeled data to train the model. In this paper, we propose to overcome these challenges by actively adapting a pre-trained model to a new task with less labeled examples. Specifically, the pre-trained model is iteratively fine tuned based on the most useful examples. The examples are actively selected based on a novel criterion, which jointly estimates the potential contribution of an instance on optimizing the feature representation as well as improving the classification model for the target task. On one hand, the pre-trained model brings plentiful information from its original task, avoiding redesign of the network architecture or training from scratch; and on the other hand, the labeling cost can be significantly reduced by active label querying. Experiments on multiple datasets and different pre-trained models demonstrate that the proposed approach can achieve cost-effective training of DNNs.