# Foreground-Action Consistency Network for Weakly Supervised Temporal Action Localization

arXiv: https://arxiv.org/abs/2108.06524 · HTML: https://mlsanity.com/paper/2108.06524/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 50 references that are in the map.
- First posted on arXiv 14 Aug 2021.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Temporal Action Detection with Structured Segment Networks](https://mlsanity.com/paper/1704.06228/) (2017)
- [Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localization](https://mlsanity.com/paper/1704.04232/) (2017)
- [Rethinking the Faster R-CNN Architecture for Temporal Action Localization](https://mlsanity.com/paper/1804.07667/) (2018)
- [R-C3D: Region Convolutional 3D Network for Temporal Activity Detection](https://mlsanity.com/paper/1703.07814/) (2017)
- [Gaussian Temporal Awareness Networks for Action Localization](https://mlsanity.com/paper/1909.03877/) (2019)
- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos](https://mlsanity.com/paper/1703.01515/) (2017)
- [End-to-end Learning of Action Detection from Frame Glimpses in Videos](https://mlsanity.com/paper/1511.06984/) (2015)
- [Weakly Supervised Action Localization by Sparse Temporal Pooling Network](https://mlsanity.com/paper/1712.05080/) (2017)
- [W-TALC: Weakly-supervised Temporal Activity Localization and Classification](https://mlsanity.com/paper/1807.10418/) (2018)
- [AutoLoc: Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1807.08333/) (2018)
- [Background Suppression Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1911.09963/) (2019)
- [Weakly-Supervised Action Localization by Generative Attention Modeling](https://mlsanity.com/paper/2003.12424/) (2020)
- [Weakly-supervised Action Localization with Background Modeling](https://mlsanity.com/paper/1908.06552/) (2019)
- [Two-Stream Consensus Network for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2010.11594/) (2020)
- [3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization](https://mlsanity.com/paper/1908.08216/) (2019)
- [Weakly Supervised Action Labeling in Videos Under Ordering Constraints](https://mlsanity.com/paper/1407.1208/) (2014)
- [Weakly-Supervised Action Localization with Expectation-Maximization Multi-Instance Learning](https://mlsanity.com/paper/2004.00163/) (2020)
- [A Hybrid Attention Mechanism for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2101.00545/) (2021)
- [Adversarial Background-Aware Loss for Weakly-supervised Temporal Activity Localization](https://mlsanity.com/paper/2007.06643/) (2020)
- [Temporal Context Network for Activity Localization in Videos](https://mlsanity.com/paper/1708.02349/) (2017)
- [Temporal Action Localization by Structured Maximal Sums](https://mlsanity.com/paper/1704.04671/) (2017)

## Built on by (most cited first, 26 of 26)

- [ASM-Loc: Action-aware Segment Modeling for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2203.15187/) (2022)
- [Fine-grained Temporal Contrastive Learning for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2203.16800/) (2022)
- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation](https://mlsanity.com/paper/2203.02925/) (2022)
- [Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature](https://mlsanity.com/paper/2303.12332/) (2023)
- [Proposal-Based Multiple Instance Learning for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2305.17861/) (2023)
- [Multi-modal Prompting for Low-Shot Temporal Action Localization](https://mlsanity.com/paper/2303.11732/) (2023)
- [Weakly-Supervised Video Anomaly Detection with Snippet Anomalous Attention](https://mlsanity.com/paper/2309.16309/) (2023)
- [Improving Weakly Supervised Temporal Action Localization by Bridging Train-Test Gap in Pseudo Labels](https://mlsanity.com/paper/2304.07978/) (2023)
- [Distilling Vision-Language Pre-training to Collaborate with Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2212.09335/) (2022)
- [Boosting Weakly-Supervised Temporal Action Localization with Text Information](https://mlsanity.com/paper/2305.00607/) (2023)
- [Revisiting Foreground and Background Separation in Weakly-supervised Temporal Action Localization: A Clustering-based Approach](https://mlsanity.com/paper/2312.14138/) (2023)
- [DDG-Net: Discriminability-Driven Graph Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2307.16415/) (2023)
- [Learning to Discriminate Information for Online Action Detection: Analysis and Application](https://mlsanity.com/paper/2109.03393/) (2021)
- [Weakly-Supervised Temporal Action Localization by Progressive Complementary Learning](https://mlsanity.com/paper/2206.11011/) (2022)
- [Forcing the Whole Video as Background: An Adversarial Learning Strategy for Weakly Temporal Action Localization](https://mlsanity.com/paper/2207.06659/) (2022)
- [Slow Motion Matters: A Slow Motion Enhanced Network for Weakly Supervised Temporal Action Localization](https://mlsanity.com/paper/2211.11324/) (2022)
- [Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling](https://mlsanity.com/paper/2308.09946/) (2023)
- [Cross-Video Contextual Knowledge Exploration and Exploitation for Ambiguity Reduction in Weakly Supervised Temporal Action Localization](https://mlsanity.com/paper/2308.12609/) (2023)
- [STAT: Towards Generalizable Temporal Action Localization](https://mlsanity.com/paper/2404.13311/) (2024)
- [Full-Stage Pseudo Label Quality Enhancement for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2407.08971/) (2024)
- [Convex Combination Consistency between Neighbors for Weakly-supervised Action Localization](https://mlsanity.com/paper/2205.00400/) (2022)
- [Weakly-Supervised Temporal Action Localization with Bidirectional Semantic Consistency Constraint](https://mlsanity.com/paper/2304.12616/) (2023)
- [Proposal-based Temporal Action Localization with Point-level Supervision](https://mlsanity.com/paper/2310.05511/) (2023)
- [BID: Boundary-Interior Decoding for Unsupervised Temporal Action Localization Pre-Trainin](https://mlsanity.com/paper/2403.07354/) (2024)
- [Video-Specific Query-Key Attention Modeling for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2305.04186/) (2023)

## Abstract (arXiv)

As a challenging task of high-level video understanding, weakly supervised temporal action localization has been attracting increasing attention. With only video annotations, most existing methods seek to handle this task with a localization-by-classification framework, which generally adopts a selector to select snippets of high probabilities of actions or namely the foreground. Nevertheless, the existing foreground selection strategies have a major limitation of only considering the unilateral relation from foreground to actions, which cannot guarantee the foreground-action consistency. In this paper, we present a framework named FAC-Net based on the I3D backbone, on which three branches are appended, named class-wise foreground classification branch, class-agnostic attention branch and multiple instance learning branch. First, our class-wise foreground classification branch regularizes the relation between actions and foreground to maximize the foreground-background separation. Besides, the class-agnostic attention branch and multiple instance learning branch are adopted to regularize the foreground-action consistency and help to learn a meaningful foreground classifier. Within each branch, we introduce a hybrid attention mechanism, which calculates multiple attention scores for each snippet, to focus on both discriminative and less-discriminative snippets to capture the full action boundaries. Experimental results on THUMOS14 and ActivityNet1.3 demonstrate the state-of-the-art performance of our method. Our code is available at https://github.com/LeonHLJ/FAC-Net.