# Scale Matters: Temporal Scale Aggregation Network for Precise Action Localization in Untrimmed Videos

arXiv: https://arxiv.org/abs/1908.00707 · HTML: https://mlsanity.com/paper/1908.00707/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Aug 2019.

## Built on by (most cited first, 17 of 17)

- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [DCAN: Improving Temporal Action Detection via Dual Context Aggregation](https://mlsanity.com/paper/2112.03612/) (2021)
- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognition](https://mlsanity.com/paper/2304.05088/) (2023)
- [TemporalMaxer: Maximize Temporal Context with only Max Pooling for Temporal Action Localization](https://mlsanity.com/paper/2303.09055/) (2023)
- [Learning to Refactor Action and Co-occurrence Features for Temporal Action Localization](https://mlsanity.com/paper/2206.11493/) (2022)
- [An Efficient Spatio-Temporal Pyramid Transformer for Action Detection](https://mlsanity.com/paper/2207.10448/) (2022)
- [Action Detection via an Image Diffusion Process](https://mlsanity.com/paper/2404.01051/) (2024)
- [Temporal Action Localization for Inertial-based Human Activity Recognition](https://mlsanity.com/paper/2311.15831/) (2023)
- [Open-Vocabulary Temporal Action Localization using Multimodal Guidance](https://mlsanity.com/paper/2406.15556/) (2024)
- [LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localization](https://mlsanity.com/paper/2404.01282/) (2024)
- [Pyramid Region-based Slot Attention Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/2206.10095/) (2022)
- [ContextDet: Temporal Action Detection with Adaptive Context Aggregation](https://mlsanity.com/paper/2410.15279/) (2024)
- [Temporal Action Localization Using Gated Recurrent Units](https://mlsanity.com/paper/2108.03375/) (2021)
- [Benchmarking Data Efficiency and Computational Efficiency of Temporal Action Localization Models](https://mlsanity.com/paper/2308.13082/) (2023)
- [Multi-Resolution Audio-Visual Feature Fusion for Temporal Action Localization](https://mlsanity.com/paper/2310.03456/) (2023)
- [Progression-Guided Temporal Action Detection in Videos](https://mlsanity.com/paper/2308.09268/) (2023)

## Abstract (arXiv)

Temporal action localization is a recently-emerging task, aiming to localize video segments from untrimmed videos that contain specific actions. Despite the remarkable recent progress, most two-stage action localization methods still suffer from imprecise temporal boundaries of action proposals. This work proposes a novel integrated temporal scale aggregation network (TSA-Net). Our main insight is that ensembling convolution filters with different dilation rates can effectively enlarge the receptive field with low computational cost, which inspires us to devise multi-dilation temporal convolution (MDC) block. Furthermore, to tackle video action instances with different durations, TSA-Net consists of multiple branches of sub-networks. Each of them adopts stacked MDC blocks with different dilation parameters, accomplishing a temporal receptive field specially optimized for specific-duration actions. We follow the formulation of boundary point detection, novelly detecting three kinds of critical points (ie, starting / mid-point / ending) and pairing them for proposal generation. Comprehensive evaluations are conducted on two challenging video benchmarks, THUMOS14 and ActivityNet-1.3. Our proposed TSA-Net demonstrates clear and consistent better performances and re-calibrates new state-of-the-art on both benchmarks. For example, our new record on THUMOS14 is 46.9% while the previous best is 42.8% under mAP@0.5.