# Active Learning for Video Classification with Frame Level Queries

arXiv: https://arxiv.org/abs/2307.05587 · HTML: https://mlsanity.com/paper/2307.05587/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 10 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Adversarial Active Learning for Deep Networks: a Margin Based Approach](https://mlsanity.com/paper/1802.09841/) (2018)
- [Deep Active Learning: Unified and Principled Method for Query and Training](https://mlsanity.com/paper/1911.09162/) (2019)
- [State-Relabeling Adversarial Active Learning](https://mlsanity.com/paper/2004.04943/) (2020)
- [Adversarial Sampling for Active Learning](https://mlsanity.com/paper/1808.06671/) (2018)
- [Cost-Effective Active Learning for Melanoma Segmentation](https://mlsanity.com/paper/1711.09168/) (2017)
- [Active Reinforcement Learning: Observing Rewards at a Cost](https://mlsanity.com/paper/2011.06709/) (2020)
- [Deep Active Learning with a Neural Architecture Search](https://mlsanity.com/paper/1811.07579/) (2018)
- [Matrix completion with queries](https://mlsanity.com/paper/1705.00399/) (2017)
- [Active Learning for Video Description With Cluster-Regularized Ensemble Ranking](https://mlsanity.com/paper/2007.13913/) (2020)
- [Active Learning in Video Tracking](https://mlsanity.com/paper/1912.12557/) (2019)

## Built on by (most cited first, 2 of 2)

- [A Review of Machine Learning Methods Applied to Video Analysis Systems](https://mlsanity.com/paper/2312.05352/) (2023)
- [Probing Association Instability with Track-State Perturbations for Clip-Level Active Learning in Query-Propagation Multi-Object Tracking](https://mlsanity.com/paper/2608.17224/) (2026)

## Abstract (arXiv)

Deep learning algorithms have pushed the boundaries of computer vision research and have depicted commendable performance in a variety of applications. However, training a robust deep neural network necessitates a large amount of labeled training data, acquiring which involves significant time and human effort. This problem is even more serious for an application like video classification, where a human annotator has to watch an entire video end-to-end to furnish a label. Active learning algorithms automatically identify the most informative samples from large amounts of unlabeled data; this tremendously reduces the human annotation effort in inducing a machine learning model, as only the few samples that are identified by the algorithm, need to be labeled manually. In this paper, we propose a novel active learning framework for video classification, with the goal of further reducing the labeling onus on the human annotators. Our framework identifies a batch of exemplar videos, together with a set of informative frames for each video; the human annotator needs to merely review the frames and provide a label for each video. This involves much less manual work than watching the complete video to come up with a label. We formulate a criterion based on uncertainty and diversity to identify the informative videos and exploit representative sampling techniques to extract a set of exemplar frames from each video. To the best of our knowledge, this is the first research effort to develop an active learning framework for video classification, where the annotators need to inspect only a few frames to produce a label, rather than watching the end-to-end video.