# Temporal Action Detection with Multi-level Supervision

arXiv: https://arxiv.org/abs/2011.11893 · HTML: https://mlsanity.com/paper/2011.11893/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [SimDA: Simple Diffusion Adapter for Efficient Video Generation](https://mlsanity.com/paper/2308.09710/) (2023)
- [SVFormer: Semi-supervised Video Transformer for Action Recognition](https://mlsanity.com/paper/2211.13222/) (2022)
- [Implicit Temporal Modeling with Learnable Alignment for Video Recognition](https://mlsanity.com/paper/2304.10465/) (2023)
- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [Semi-Supervised Temporal Action Detection with Proposal-Free Masking](https://mlsanity.com/paper/2207.07059/) (2022)
- [OSCAR: Object Status and Contextual Awareness for Recipes to Support Non-Visual Cooking](https://mlsanity.com/paper/2503.05962/) (2025)
- [Exploring Object Status Recognition for Recipe Progress Tracking in Non-Visual Cooking](https://mlsanity.com/paper/2507.03330/) (2025)
- [Towards Adaptive Pseudo-label Learning for Semi-Supervised Temporal Action Localization](https://mlsanity.com/paper/2407.07673/) (2024)
- [Enabling Weakly-Supervised Temporal Action Localization from On-Device Learning of the Video Stream](https://mlsanity.com/paper/2208.12673/) (2022)

## Abstract (arXiv)

Training temporal action detection in videos requires large amounts of labeled data, yet such annotation is expensive to collect. Incorporating unlabeled or weakly-labeled data to train action detection model could help reduce annotation cost. In this work, we first introduce the Semi-supervised Action Detection (SSAD) task with a mixture of labeled and unlabeled data and analyze different types of errors in the proposed SSAD baselines which are directly adapted from the semi-supervised classification task. To alleviate the main error of action incompleteness (i.e., missing parts of actions) in SSAD baselines, we further design an unsupervised foreground attention (UFA) module utilizing the "independence" between foreground and background motion. Then we incorporate weakly-labeled data into SSAD and propose Omni-supervised Action Detection (OSAD) with three levels of supervision. An information bottleneck (IB) suppressing the scene information in non-action frames while preserving the action information is designed to help overcome the accompanying action-context confusion problem in OSAD baselines. We extensively benchmark against the baselines for SSAD and OSAD on our created data splits in THUMOS14 and ActivityNet1.2, and demonstrate the effectiveness of the proposed UFA and IB methods. Lastly, the benefit of our full OSAD-IB model under limited annotation budgets is shown by exploring the optimal annotation strategy for labeled, unlabeled and weakly-labeled data.