# Accurate Temporal Action Proposal Generation with Relation-Aware Pyramid Network

arXiv: https://arxiv.org/abs/2003.04145 · HTML: https://mlsanity.com/paper/2003.04145/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2020.

## Built on by (most cited first, 28 of 28)

- [Relaxed Transformer Decoders for Direct Action Proposal Generation](https://mlsanity.com/paper/2102.01894/) (2021)
- [Temporal Context Aggregation Network for Temporal Action Proposal Refinement](https://mlsanity.com/paper/2103.13141/) (2021)
- [TALLFormer: Temporal Action Localization with a Long-memory Transformer](https://mlsanity.com/paper/2204.01680/) (2022)
- [Class Semantics-based Attention for Action Detection](https://mlsanity.com/paper/2109.02613/) (2021)
- [DCAN: Improving Temporal Action Detection via Dual Context Aggregation](https://mlsanity.com/paper/2112.03612/) (2021)
- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [BasicTAD: an Astounding RGB-Only Baseline for Temporal Action Detection](https://mlsanity.com/paper/2205.02717/) (2022)
- [TemporalMaxer: Maximize Temporal Context with only Max Pooling for Temporal Action Localization](https://mlsanity.com/paper/2303.09055/) (2023)
- [Relation-aware Video Reading Comprehension for Temporal Language Grounding](https://mlsanity.com/paper/2110.05717/) (2021)
- [Aggregated Multi-GANs for Controlled 3D Human Motion Prediction](https://mlsanity.com/paper/2103.09755/) (2021)
- [PointTAD: Multi-Label Temporal Action Detection with Learnable Query Points](https://mlsanity.com/paper/2210.11035/) (2022)
- [Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization](https://mlsanity.com/paper/2103.15233/) (2021)
- [Dual DETRs for Multi-Label Temporal Action Detection](https://mlsanity.com/paper/2404.00653/) (2024)
- [Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2104.02357/) (2021)
- [Learning to Refactor Action and Co-occurrence Features for Temporal Action Localization](https://mlsanity.com/paper/2206.11493/) (2022)
- [Temporal Action Proposal Generation with Transformers](https://mlsanity.com/paper/2105.12043/) (2021)
- [RGB Stream Is Enough for Temporal Action Detection](https://mlsanity.com/paper/2107.04362/) (2021)
- [Temporal Action Proposal Generation with Background Constraint](https://mlsanity.com/paper/2112.07984/) (2021)
- [Learning from Untrimmed Videos: Self-Supervised Video Representation Learning with Hierarchical Consistency](https://mlsanity.com/paper/2204.03017/) (2022)
- [AOE-Net: Entities Interactions Modeling with Adaptive Attention Mechanism for Temporal Action Proposals Generation](https://mlsanity.com/paper/2210.02578/) (2022)
- [You Need to Read Again: Multi-granularity Perception Network for Moment Retrieval in Videos](https://mlsanity.com/paper/2205.12886/) (2022)
- [Temporal Perceiver: A General Architecture for Arbitrary Boundary Detection](https://mlsanity.com/paper/2203.00307/) (2022)
- [HiCo: Hierarchical Contrastive Learning for Ultrasound Video Model Pretraining](https://mlsanity.com/paper/2210.04477/) (2022)
- [Towards High-Quality Temporal Action Detection with Sparse Proposals](https://mlsanity.com/paper/2109.08847/) (2021)
- [PcmNet: Position-Sensitive Context Modeling Network for Temporal Action Localization](https://mlsanity.com/paper/2103.05270/) (2021)
- [Transferable Knowledge-Based Multi-Granularity Aggregation Network for Temporal Action Localization: Submission to ActivityNet Challenge 2021](https://mlsanity.com/paper/2107.12618/) (2021)
- [Faster Learning of Temporal Action Proposal via Sparse Multilevel Boundary Generator](https://mlsanity.com/paper/2303.03166/) (2023)
- [Semantic Video Moments Retrieval at Scale: A New Task and a Baseline](https://mlsanity.com/paper/2210.08389/) (2022)

## Abstract (arXiv)

Accurate temporal action proposals play an important role in detecting actions from untrimmed videos. The existing approaches have difficulties in capturing global contextual information and simultaneously localizing actions with different durations. To this end, we propose a Relation-aware pyramid Network (RapNet) to generate highly accurate temporal action proposals. In RapNet, a novel relation-aware module is introduced to exploit bi-directional long-range relations between local features for context distilling. This embedded module enhances the RapNet in terms of its multi-granularity temporal proposal generation ability, given predefined anchor boxes. We further introduce a two-stage adjustment scheme to refine the proposal boundaries and measure their confidence in containing an action with snippet-level actionness. Extensive experiments on the challenging ActivityNet and THUMOS14 benchmarks demonstrate our RapNet generates superior accurate proposals over the existing state-of-the-art methods.