# Active Learning for Video Description With Cluster-Regularized Ensemble Ranking

arXiv: https://arxiv.org/abs/2007.13913 · HTML: https://mlsanity.com/paper/2007.13913/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jul 2020.

## Built on by (most cited first, 5 of 5)

- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [Correlation-aware active learning for surgery video segmentation](https://mlsanity.com/paper/2311.08811/) (2023)
- [Active Learning for Video Classification with Frame Level Queries](https://mlsanity.com/paper/2307.05587/) (2023)
- [Active Learning for Fine-Grained Sketch-Based Image Retrieval](https://mlsanity.com/paper/2309.08743/) (2023)
- [MAViC: Multimodal Active Learning for Video Captioning](https://mlsanity.com/paper/2212.11109/) (2022)

## Abstract (arXiv)

Automatic video captioning aims to train models to generate text descriptions for all segments in a video, however, the most effective approaches require large amounts of manual annotation which is slow and expensive. Active learning is a promising way to efficiently build a training set for video captioning tasks while reducing the need to manually label uninformative examples. In this work we both explore various active learning approaches for automatic video captioning and show that a cluster-regularized ensemble strategy provides the best active learning approach to efficiently gather training sets for video captioning. We evaluate our approaches on the MSR-VTT and LSMDC datasets using both transformer and LSTM based captioning models and show that our novel strategy can achieve high performance while using up to 60% fewer training data than the strong state of the art baselines.