# Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization

arXiv: https://arxiv.org/abs/2104.02357 · HTML: https://mlsanity.com/paper/2104.02357/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 67 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 30 of 42)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Temporal Action Detection with Structured Segment Networks](https://mlsanity.com/paper/1704.06228/) (2017)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019)
- [Rethinking the Faster R-CNN Architecture for Temporal Action Localization](https://mlsanity.com/paper/1804.07667/) (2018)
- [R-C3D: Region Convolutional 3D Network for Temporal Activity Detection](https://mlsanity.com/paper/1703.07814/) (2017)
- [Gaussian Temporal Awareness Networks for Action Localization](https://mlsanity.com/paper/1909.03877/) (2019)
- [SCSampler: Sampling Salient Clips from Video for Efficient Action Recognition](https://mlsanity.com/paper/1904.04289/) (2019)
- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [Single Shot Temporal Action Detection](https://mlsanity.com/paper/1710.06236/) (2017)
- [TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals](https://mlsanity.com/paper/1703.06189/) (2017)
- [CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos](https://mlsanity.com/paper/1703.01515/) (2017)
- [Weakly Supervised Action Localization by Sparse Temporal Pooling Network](https://mlsanity.com/paper/1712.05080/) (2017)
- [Fast Learning of Temporal Action Proposal via Dense Boundary Generator](https://mlsanity.com/paper/1911.04127/) (2019)
- [Boundary Content Graph Neural Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/2008.01432/) (2020)
- [W-TALC: Weakly-supervised Temporal Activity Localization and Classification](https://mlsanity.com/paper/1807.10418/) (2018)
- [Multi-granularity Generator for Temporal Action Proposal](https://mlsanity.com/paper/1811.11524/) (2018)
- [AutoLoc: Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1807.08333/) (2018)
- [Revisiting Anchor Mechanisms for Temporal Action Localization](https://mlsanity.com/paper/2008.09837/) (2020)
- [Background Suppression Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1911.09963/) (2019)
- [Weakly-Supervised Action Localization by Generative Attention Modeling](https://mlsanity.com/paper/2003.12424/) (2020)
- [Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks](https://mlsanity.com/paper/1809.03355/) (2018)
- [Weakly-supervised Action Localization with Background Modeling](https://mlsanity.com/paper/1908.06552/) (2019)
- [Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2010.11594/) (2020)
- [3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization](https://mlsanity.com/paper/1908.08216/) (2019)

## Built on by (most cited first, 18 of 18)

- [Prompting Visual-Language Models for Efficient Video Understanding](https://mlsanity.com/paper/2112.04478/) (2021)
- [FOLDER: Accelerating Multi-modal Large Language Models with Enhanced Performance](https://mlsanity.com/paper/2501.02430/) (2025)
- [AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation](https://mlsanity.com/paper/2309.00096/) (2023)
- [Multi-modal Prompting for Low-Shot Temporal Action Localization](https://mlsanity.com/paper/2303.11732/) (2023)
- [Constraint and Union for Partially-Supervised Temporal Sentence Grounding](https://mlsanity.com/paper/2302.09850/) (2023)
- [Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Models](https://mlsanity.com/paper/2407.11717/) (2024)
- [Distilling Vision-Language Pre-training to Collaborate with Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2212.09335/) (2022)
- [DENOISER: Rethinking the Robustness for Open-Vocabulary Action Recognition](https://mlsanity.com/paper/2404.14890/) (2024)
- [DDG-Net: Discriminability-Driven Graph Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2307.16415/) (2023)
- [Cross-Video Contextual Knowledge Exploration and Exploitation for Ambiguity Reduction in Weakly Supervised Temporal Action Localization](https://mlsanity.com/paper/2308.12609/) (2023)
- [Contrast-Unity for Partially-Supervised Temporal Sentence Grounding](https://mlsanity.com/paper/2502.12917/) (2025)
- [Explore More, Learn Better: Parallel MLLM Embeddings under Mutual Information Minimization](https://mlsanity.com/paper/2511.01588/) (2025)
- [Towards Weakly Supervised End-to-end Learning for Long-video Action Recognition](https://mlsanity.com/paper/2311.17118/) (2023)
- [Squeeze Out Tokens from Sample for Finer-Grained Data Governance](https://mlsanity.com/paper/2503.14559/) (2025)
- [Wave-Particle (Continuous-Discrete) Dualistic Visual Tokenization for Unified Understanding and Generation](https://mlsanity.com/paper/2511.01593/) (2025)
- [Unbiased Semantic Decoding with Vision Foundation Models for Few-shot Segmentation](https://mlsanity.com/paper/2511.15118/) (2025)
- [Probabilistic Temporal Masked Attention for Cross-view Online Action Detection](https://mlsanity.com/paper/2508.17025/) (2025)
- [Beyond Visual Cues: Leveraging General Semantics as Support for Few-Shot Segmentation](https://mlsanity.com/paper/2511.16435/) (2025)

## Abstract (arXiv)

Weakly-supervised temporal action localization aims to localize actions in untrimmed videos with only video-level action category labels. Most of previous methods ignore the incompleteness issue of Class Activation Sequences (CAS), suffering from trivial localization results. To solve this issue, we introduce an adaptive mutual supervision framework (AMS) with two branches, where the base branch adopts CAS to localize the most discriminative action regions, while the supplementary branch localizes the less discriminative action regions through a novel adaptive sampler. The adaptive sampler dynamically updates the input of the supplementary branch with a sampling weight sequence negatively correlated with the CAS from the base branch, thereby prompting the supplementary branch to localize the action regions underestimated by the base branch. To promote mutual enhancement between these two branches, we construct mutual location supervision. Each branch leverages location pseudo-labels generated from the other branch as localization supervision. By alternately optimizing the two branches in multiple iterations, we progressively complete action regions. Extensive experiments on THUMOS14 and ActivityNet1.2 demonstrate that the proposed AMS method significantly outperforms the state-of-the-art methods.