# Gaussian Temporal Awareness Networks for Action Localization

arXiv: https://arxiv.org/abs/1909.03877 · HTML: https://mlsanity.com/paper/1909.03877/

## Key facts

- Cited by 135 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 30 of 135)

- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [Learning Salient Boundary Feature for Anchor-free Temporal Action Localization](https://mlsanity.com/paper/2103.13137/) (2021)
- [End-to-end Temporal Action Detection with Transformer](https://mlsanity.com/paper/2106.10271/) (2021)
- [TriDet: Temporal Action Detection with Relative Boundary Modeling](https://mlsanity.com/paper/2303.07347/) (2023)
- [TALLFormer: Temporal Action Localization with a Long-memory Transformer](https://mlsanity.com/paper/2204.01680/) (2022)
- [Object-Region Video Transformers](https://mlsanity.com/paper/2110.06915/) (2021)
- [CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning](https://mlsanity.com/paper/2103.16392/) (2021)
- [UnLoc: A Unified Framework for Video Localization Tasks](https://mlsanity.com/paper/2308.11062/) (2023)
- [Generic Event Boundary Detection: A Benchmark for Event Segmentation](https://mlsanity.com/paper/2101.10511/) (2021)
- [Zero-Shot Temporal Action Detection via Vision-Language Prompting](https://mlsanity.com/paper/2207.08184/) (2022)
- [Enriching Local and Global Contexts for Temporal Action Localization](https://mlsanity.com/paper/2107.12960/) (2021)
- [VideoStudio: Generating Consistent-Content and Multi-Scene Videos](https://mlsanity.com/paper/2401.01256/) (2024)
- [FineAction: A Fine-Grained Video Dataset for Temporal Action Localization](https://mlsanity.com/paper/2105.11107/) (2021)
- [ReAct: Temporal Action Detection with Relational Queries](https://mlsanity.com/paper/2207.07097/) (2022)
- [Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baseline](https://mlsanity.com/paper/2303.12930/) (2023)
- [ASM-Loc: Action-aware Segment Modeling for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2203.15187/) (2022)
- [An Empirical Study of End-to-End Temporal Action Detection](https://mlsanity.com/paper/2204.02932/) (2022)
- [Class Semantics-based Attention for Action Detection](https://mlsanity.com/paper/2109.02613/) (2021)
- [DCAN: Improving Temporal Action Detection via Dual Context Aggregation](https://mlsanity.com/paper/2112.03612/) (2021)
- [Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Grounding](https://mlsanity.com/paper/2203.02966/) (2022)
- [Fine-grained Temporal Contrastive Learning for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2203.16800/) (2022)
- [Modeling Multi-Label Action Dependencies for Temporal Action Localization](https://mlsanity.com/paper/2103.03027/) (2021)
- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation](https://mlsanity.com/paper/2203.02925/) (2022)
- [WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognition](https://mlsanity.com/paper/2304.05088/) (2023)
- [Unsupervised Pre-training for Temporal Action Localization Tasks](https://mlsanity.com/paper/2203.13609/) (2022)
- [Proposal-Free Temporal Action Detection via Global Segmentation Mask Learning](https://mlsanity.com/paper/2207.06580/) (2022)
- [BasicTAD: an Astounding RGB-Only Baseline for Temporal Action Detection](https://mlsanity.com/paper/2205.02717/) (2022)
- [TemporalMaxer: Maximize Temporal Context with only Max Pooling for Temporal Action Localization](https://mlsanity.com/paper/2303.09055/) (2023)
- [Action Sensitivity Learning for Temporal Action Localization](https://mlsanity.com/paper/2305.15701/) (2023)

## Abstract (arXiv)

Temporally localizing actions in a video is a fundamental challenge in video understanding. Most existing approaches have often drawn inspiration from image object detection and extended the advances, e.g., SSD and Faster R-CNN, to produce temporal locations of an action in a 1D sequence. Nevertheless, the results can suffer from robustness problem due to the design of predetermined temporal scales, which overlooks the temporal structure of an action and limits the utility on detecting actions with complex variations. In this paper, we propose to address the problem by introducing Gaussian kernels to dynamically optimize temporal scale of each action proposal. Specifically, we present Gaussian Temporal Awareness Networks (GTAN) --- a new architecture that novelly integrates the exploitation of temporal structure into an one-stage action localization framework. Technically, GTAN models the temporal structure through learning a set of Gaussian kernels, each for a cell in the feature maps. Each Gaussian kernel corresponds to a particular interval of an action proposal and a mixture of Gaussian kernels could further characterize action proposals with various length. Moreover, the values in each Gaussian curve reflect the contextual contributions to the localization of an action proposal. Extensive experiments are conducted on both THUMOS14 and ActivityNet v1.3 datasets, and superior results are reported when comparing to state-of-the-art approaches. More remarkably, GTAN achieves 1.9% and 1.1% improvements in mAP on testing set of the two datasets.