# Factors in Finetuning Deep Model for object detection

arXiv: https://arxiv.org/abs/1601.05150 · HTML: https://mlsanity.com/paper/1601.05150/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2016.

## Built on by (most cited first, 30 of 53)

- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Parametric Contrastive Learning](https://mlsanity.com/paper/2107.12028/) (2021)
- [SynFace: Face Recognition with Synthetic Data](https://mlsanity.com/paper/2108.07960/) (2021)
- [Distributional Robustness Loss for Long-tail Learning](https://mlsanity.com/paper/2104.03066/) (2021)
- [V3Det: Vast Vocabulary Visual Detection Dataset](https://mlsanity.com/paper/2304.03752/) (2023)
- [A Survey on Long-Tailed Visual Recognition](https://mlsanity.com/paper/2205.13775/) (2022)
- [Label-Imbalanced and Group-Sensitive Classification under Overparameterization](https://mlsanity.com/paper/2103.01550/) (2021)
- [A Simple Long-Tailed Recognition Baseline via Vision-Language Model](https://mlsanity.com/paper/2111.14745/) (2021)
- [Class-Balancing Diffusion Models](https://mlsanity.com/paper/2305.00562/) (2023)
- [Generalized Parametric Contrastive Learning](https://mlsanity.com/paper/2209.12400/) (2022)
- [Injecting Semantic Concepts into End-to-End Image Captioning](https://mlsanity.com/paper/2112.05230/) (2021)
- [Semi-Supervised Object Detection with Adaptive Class-Rebalancing Self-Training](https://mlsanity.com/paper/2107.05031/) (2021)
- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [Procrustean Training for Imbalanced Deep Learning](https://mlsanity.com/paper/2104.01769/) (2021)
- [Open Long-Tailed Recognition in a Dynamic World](https://mlsanity.com/paper/2208.08349/) (2022)
- [Evaluating Distributional Distortion in Neural Language Modeling](https://mlsanity.com/paper/2203.12788/) (2022)
- [Simplifying Neural Network Training Under Class Imbalance](https://mlsanity.com/paper/2312.02517/) (2023)
- [ResNorm: Tackling Long-tailed Degree Distribution Issue in Graph Neural Networks via Normalization](https://mlsanity.com/paper/2206.08181/) (2022)
- [LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-tailed Multi-Label Visual Recognition](https://mlsanity.com/paper/2305.04536/) (2023)
- [Latent-based Diffusion Model for Long-tailed Recognition](https://mlsanity.com/paper/2404.04517/) (2024)
- [Improving Large Models with Small models: Lower Costs and Better Performance](https://mlsanity.com/paper/2406.15471/) (2024)
- [A Comprehensive Survey on Imbalanced Data Learning](https://mlsanity.com/paper/2502.08960/) (2025)
- [Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifolds](https://mlsanity.com/paper/2303.12307/) (2023)
- [SuperDisco: Super-Class Discovery Improves Visual Recognition for the Long-Tail](https://mlsanity.com/paper/2304.00101/) (2023)
- [Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation](https://mlsanity.com/paper/2103.14269/) (2021)
- [Long-tailed Distribution Adaptation](https://mlsanity.com/paper/2110.02686/) (2021)
- [Zone Evaluation: Revealing Spatial Bias in Object Detection](https://mlsanity.com/paper/2310.13215/) (2023)
- [Language bias in Visual Question Answering: A Survey and Taxonomy](https://mlsanity.com/paper/2111.08531/) (2021)
- [Towards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipation](https://mlsanity.com/paper/2411.13059/) (2024)

## Abstract (arXiv)

Finetuning from a pretrained deep model is found to yield state-of-the-art performance for many vision tasks. This paper investigates many factors that influence the performance in finetuning for object detection. There is a long-tailed distribution of sample numbers for classes in object detection. Our analysis and empirical results show that classes with more samples have higher impact on the feature learning. And it is better to make the sample number more uniform across classes. Generic object detection can be considered as multiple equally important tasks. Detection of each class is a task. These classes/tasks have their individuality in discriminative visual appearance representation. Taking this individuality into account, we cluster objects into visually similar class groups and learn deep representations for these groups separately. A hierarchical feature learning scheme is proposed. In this scheme, the knowledge from the group with large number of classes is transferred for learning features in its sub-groups. Finetuned on the GoogLeNet model, experimental results show 4.7% absolute mAP improvement of our approach on the ImageNet object detection dataset without increasing much computational cost at the testing stage.