# Semi-supervised Active Learning for Video Action Detection

arXiv: https://arxiv.org/abs/2312.07169 · HTML: https://mlsanity.com/paper/2312.07169/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 68 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 30 of 47)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Tracking Emerges by Colorizing Videos](https://mlsanity.com/paper/1806.09594/) (2018)
- [Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration](https://mlsanity.com/paper/1707.02920/) (2017)
- [Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization](https://mlsanity.com/paper/2006.07976/) (2020)
- [Actor-Centric Relation Network](https://mlsanity.com/paper/1807.10982/) (2018)
- [Self-Supervised Learning for Videos: A Survey](https://mlsanity.com/paper/2207.00419/) (2022)
- [Action Tubelet Detector for Spatio-Temporal Action Localization](https://mlsanity.com/paper/1705.01861/) (2017)
- [Actions as Moving Points](https://mlsanity.com/paper/2001.04608/) (2020)
- [TubeR: Tubelet Transformer for Video Action Detection](https://mlsanity.com/paper/2104.00969/) (2021)
- [STEP: Spatio-Temporal Progressive Learning for Video Action Detection](https://mlsanity.com/paper/1904.09288/) (2019)

## Built on by (most cited first, 10 of 10)

- [STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding](https://mlsanity.com/paper/2502.20678/) (2025)
- [Contextual Self-paced Learning for Weakly Supervised Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2501.17053/) (2025)
- [A Large-Scale Analysis on Contextual Self-Supervised Video Representation Learning](https://mlsanity.com/paper/2504.06153/) (2025)
- [IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction](https://mlsanity.com/paper/2605.01666/) (2026)
- [PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models](https://mlsanity.com/paper/2506.00910/) (2025)
- [GaitCrafter: Diffusion Model for Biometric Preserving Gait Synthesis](https://mlsanity.com/paper/2508.13300/) (2025)
- [RobustGait: Robustness Analysis for Appearance Based Gait Recognition](https://mlsanity.com/paper/2511.13065/) (2025)
- [Denoise and Align: Diffusion-Driven Foreground Knowledge Prompting for Open-Vocabulary Temporal Action Detection](https://mlsanity.com/paper/2604.18313/) (2026)
- [VISTA: Video Interaction Spatio-Temporal Analysis Benchmark](https://mlsanity.com/paper/2605.01391/) (2026)
- [Probing Association Instability with Track-State Perturbations for Clip-Level Active Learning in Query-Propagation Multi-Object Tracking](https://mlsanity.com/paper/2608.17224/) (2026)

## Abstract (arXiv)

In this work, we focus on label efficient learning for video action detection. We develop a novel semi-supervised active learning approach which utilizes both labeled as well as unlabeled data along with informative sample selection for action detection. Video action detection requires spatio-temporal localization along with classification, which poses several challenges for both active learning informative sample selection as well as semi-supervised learning pseudo label generation. First, we propose NoiseAug, a simple augmentation strategy which effectively selects informative samples for video action detection. Next, we propose fft-attention, a novel technique based on high-pass filtering which enables effective utilization of pseudo label for SSL in video action detection by emphasizing on relevant activity region within a video. We evaluate the proposed approach on three different benchmark datasets, UCF-101-24, JHMDB-21, and Youtube-VOS. First, we demonstrate its effectiveness on video action detection where the proposed approach outperforms prior works in semi-supervised and weakly-supervised learning along with several baseline approaches in both UCF101-24 and JHMDB-21. Next, we also show its effectiveness on Youtube-VOS for video object segmentation demonstrating its generalization capability for other dense prediction tasks in videos. The code and models is publicly available at: urlhttps://github.com/AKASH2907/semi-sup-active-learning.