# Boosting Semi-Supervised Temporal Action Localization by Learning from Non-Target Classes

arXiv: https://arxiv.org/abs/2403.11189 · HTML: https://mlsanity.com/paper/2403.11189/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 61 references that are in the map.
- First posted on arXiv 17 Mar 2024.

## Builds on (most cited first, 30 of 37)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [Rethinking the Faster R-CNN Architecture for Temporal Action Localization](https://mlsanity.com/paper/1804.07667/) (2018)
- [Temporal Pyramid Network for Action Recognition](https://mlsanity.com/paper/2004.03548/) (2020)
- [Learning to Learn from Noisy Labeled Data](https://mlsanity.com/paper/1812.05214/) (2018)
- [Learning Salient Boundary Feature for Anchor-free Temporal Action Localization](https://mlsanity.com/paper/2103.13137/) (2021)
- [NLNL: Negative Learning for Noisy Labels](https://mlsanity.com/paper/1908.07387/) (2019)
- [End-to-end Temporal Action Detection with Transformer](https://mlsanity.com/paper/2106.10271/) (2021)
- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework](https://mlsanity.com/paper/2103.11402/) (2021)
- [Learning with Biased Complementary Labels](https://mlsanity.com/paper/1711.09535/) (2017)
- [Learning from Complementary Labels](https://mlsanity.com/paper/1705.07541/) (2017)
- [BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation](https://mlsanity.com/paper/2009.07641/) (2020)
- [CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016](https://mlsanity.com/paper/1608.00797/) (2016)
- [Multi-shot Temporal Event Localization: a Benchmark](https://mlsanity.com/paper/2012.09434/) (2020)
- [ReAct: Temporal Action Detection with Relational Queries](https://mlsanity.com/paper/2207.07097/) (2022)
- [Label Matching Semi-Supervised Object Detection](https://mlsanity.com/paper/2206.06608/) (2022)
- [Self-Supervised Learning for Semi-Supervised Temporal Action Proposal](https://mlsanity.com/paper/2104.03214/) (2021)
- [Spatio-Temporal Channel Correlation Networks for Action Classification](https://mlsanity.com/paper/1806.07754/) (2018)
- [TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection](https://mlsanity.com/paper/1905.13417/) (2019)
- [Semi-Supervised Semantic Segmentation via Gentle Teaching Assistant](https://mlsanity.com/paper/2301.07340/) (2023)
- [Graph Convolutional Module for Temporal Action Localization in Videos](https://mlsanity.com/paper/2112.00302/) (2021)
- [MAR: Masked Autoencoders for Efficient Action Recognition](https://mlsanity.com/paper/2207.11660/) (2022)

## Built on by (most cited first, 1 of 1)

- [Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer](https://mlsanity.com/paper/2504.14860/) (2025)

## Abstract (arXiv)

The crux of semi-supervised temporal action localization (SS-TAL) lies in excavating valuable information from abundant unlabeled videos. However, current approaches predominantly focus on building models that are robust to the error-prone target class (i.e, the predicted class with the highest confidence) while ignoring informative semantics within non-target classes. This paper approaches SS-TAL from a novel perspective by advocating for learning from non-target classes, transcending the conventional focus solely on the target class. The proposed approach involves partitioning the label space of the predicted class distribution into distinct subspaces: target class, positive classes, negative classes, and ambiguous classes, aiming to mine both positive and negative semantics that are absent in the target class, while excluding ambiguous classes. To this end, we first devise innovative strategies to adaptively select high-quality positive and negative classes from the label space, by modeling both the confidence and rank of a class in relation to those of the target class. Then, we introduce novel positive and negative losses designed to guide the learning process, pushing predictions closer to positive classes and away from negative classes. Finally, the positive and negative processes are integrated into a hybrid positive-negative learning framework, facilitating the utilization of non-target classes in both labeled and unlabeled videos. Experimental results on THUMOS14 and ActivityNet v1.3 demonstrate the superiority of the proposed method over prior state-of-the-art approaches.