mlsanityData to 7 Oct 2026

The Blessings of Unlabeled Background in Untrimmed Videos

arXiv 2103.13183 ↗First posted 24 Mar 2021cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Weakly-supervised Temporal Action Localization (WTAL) aims to detect the action segments with only video-level action labels in training. The key challenge is how to distinguish the action of interest segments from the background, which is unlabelled even on the video-level. While previous works treat the background as "curses", we consider it as "blessings". Specifically, we first use causal analysis to point out that the common localization errors are due to the unobserved confounder that resides ubiquitously in visual recognition. Then, we propose a Temporal Smoothing PCA-based (TS-PCA) deconfounder, which exploits the unlabelled background to model an observed substitute for the unobserved confounder, to remove the confounding effect. Note that the proposed deconfounder is model-agnostic and non-intrusive, and hence can be applied in any WTAL method without model re-designs. Through extensive experiments on four state-of-the-art WTAL methods, we show that the deconfounder can improve all of them on the public datasets: THUMOS-14 and ActivityNet-1.3.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 24 Mar 2021.

Builds on

31 of 45 references.

References · 31

  1. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  2. 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
  3. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  4. 2017Temporal Action Detection with Structured Segment Networkscited by 265
  5. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 248
  6. 2020Causal Intervention for Weakly-Supervised Semantic Segmentationcited by 229
  7. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  8. 2017Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localizationcited by 207
  9. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  10. 2017R-C3D: Region Convolutional 3D Network for Temporal Activity Detectioncited by 161
  11. 2019Gaussian Temporal Awareness Networks for Action Localizationcited by 135
  12. 2017UntrimmedNets for Weakly Supervised Action Recognition and Detectioncited by 129
19 more
  1. 2017CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videoscited by 112
  2. 2020Interventional Few-Shot Learningcited by 112
  3. 2018The Blessings of Multiple Causescited by 103
  4. 2017Weakly Supervised Action Localization by Sparse Temporal Pooling Networkcited by 100
  5. 2018W-TALC: Weakly-supervised Temporal Activity Localization and Classificationcited by 87
  6. 2018Multi-granularity Generator for Temporal Action Proposalcited by 77
  7. 2018AutoLoc: Weakly-supervised Temporal Action Localizationcited by 75
  8. 2019Two Causal Principles for Improving Visual Dialogcited by 75
  9. 2019Background Suppression Network for Weakly-supervised Temporal Action Localizationcited by 73
  10. 2018Diagnosing Error in Temporal Action Detectorscited by 71
  11. 2014Visual Causal Feature Learningcited by 59
  12. 2020Weakly-Supervised Action Localization by Generative Attention Modelingcited by 58
  13. 2019Weakly-supervised Action Localization with Background Modelingcited by 53
  14. 2020Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localizationcited by 53
  15. 20193C-Net: Category Count and Center Loss for Weakly-Supervised Action Localizationcited by 52
  16. 2018Counterfactuals uncover the modular structure of deep generative modelscited by 51
  17. 2020Adversarial Background-Aware Loss for Weakly-supervised Temporal Activity Localizationcited by 40
  18. 2018Step-by-step Erasion, One-by-one Collection: A Weakly Supervised Temporal Action Detectorcited by 27
  19. 2019Marginalized Average Attentional Network for Weakly-Supervised Learningcited by 24

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 6; 2023: 5; 2024: 1; 2025: 1.

16 papers.

Sort:

By area:

  1. 2021Deconfounded Video Moment Retrieval with Causal Interventioncited by 84
  2. 2021Learning Action Completeness from Points for Weakly-supervised Temporal Action Localizationcited by 42
  3. 2022ASM-Loc: Action-aware Segment Modeling for Weakly-Supervised Temporal Action Localizationcited by 39
  4. 2022Masked Motion Encoding for Self-Supervised Video Representation Learningcited by 39
  5. 2022Fine-grained Temporal Contrastive Learning for Weakly-supervised Temporal Action Localizationcited by 33
  6. 2022Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagationcited by 32
  7. 2023Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Featurecited by 21
  8. 2023Proposal-Based Multiple Instance Learning for Weakly-Supervised Temporal Action Localizationcited by 19
  9. 2023Improving Weakly Supervised Temporal Action Localization by Bridging Train-Test Gap in Pseudo Labelscited by 14
  10. 2021Weakly-Supervised Video Object Grounding via Causal Interventioncited by 13
  11. 2022Distilling Vision-Language Pre-training to Collaborate with Weakly-Supervised Temporal Action Localizationcited by 12
  12. 2022Post-Processing Temporal Action Detectioncited by 6
4 more
  1. 2024Full-Stage Pseudo Label Quality Enhancement for Weakly-supervised Temporal Action Localizationcited by 3
  2. 2023Weakly-Supervised Temporal Action Localization with Bidirectional Semantic Consistency Constraintcited by 2
  3. 2023Video-Specific Query-Key Attention Modeling for Weakly-Supervised Temporal Action Localizationcited by 0
  4. 2025ProTAL: A Drag-and-Link Video Programming Framework for Temporal Action Localizationcited by 0