# Proposal Relation Network for Temporal Action Detection

arXiv: https://arxiv.org/abs/2106.11812 · HTML: https://mlsanity.com/paper/2106.11812/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 21 references that are in the map.
- First posted on arXiv 20 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [A Short Note on the Kinetics-700 Human Action Dataset](https://mlsanity.com/paper/1907.06987/) (2019)
- [Video Classification with Channel-Separated Convolutional Networks](https://mlsanity.com/paper/1904.02811/) (2019)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019)
- [Temporal Context Aggregation Network for Temporal Action Proposal Refinement](https://mlsanity.com/paper/2103.13141/) (2021)
- [Boundary-sensitive Pre-training for Temporal Localization in Videos](https://mlsanity.com/paper/2011.10830/) (2020)
- [Self-Supervised Learning for Semi-Supervised Temporal Action Proposal](https://mlsanity.com/paper/2104.03214/) (2021)
- [TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection](https://mlsanity.com/paper/1905.13417/) (2019)
- [Self-supervised Motion Learning from Static Images](https://mlsanity.com/paper/2104.00240/) (2021)
- [CBR-Net: Cascade Boundary Refinement Network for Action Detection: Submission to ActivityNet Challenge 2020 (Task 1)](https://mlsanity.com/paper/2006.07526/) (2020)

## Built on by (most cited first, 18 of 18)

- [Hybrid Relation Guided Set Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2204.13423/) (2022)
- [MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition](https://mlsanity.com/paper/2304.00946/) (2023)
- [360+x: A Panoptic Multi-modal Scene Understanding Dataset](https://mlsanity.com/paper/2404.00989/) (2024)
- [Action Sensitivity Learning for Temporal Action Localization](https://mlsanity.com/paper/2305.15701/) (2023)
- [SVIP: Sequence VerIfication for Procedures in Videos](https://mlsanity.com/paper/2112.06447/) (2021)
- [HR-Pro: Point-supervised Temporal Action Localization via Hierarchical Reliability Propagation](https://mlsanity.com/paper/2308.12608/) (2023)
- [UniMD: Towards Unifying Moment Retrieval and Temporal Action Detection](https://mlsanity.com/paper/2404.04933/) (2024)
- [VideoGLUE: Video General Understanding Evaluation of Foundation Models](https://mlsanity.com/paper/2307.03166/) (2023)
- [Faster-TAD: Towards Temporal Action Detection with Proposal Generation and Classification in a Unified Network](https://mlsanity.com/paper/2204.02674/) (2022)
- [Trajectory-aligned Space-time Tokens for Few-shot Action Recognition](https://mlsanity.com/paper/2407.18249/) (2024)
- [Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognition](https://mlsanity.com/paper/2508.03695/) (2025)
- [Programmatic Concept Learning for Human Motion Description and Synthesis](https://mlsanity.com/paper/2206.13502/) (2022)
- [DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in Long Videos](https://mlsanity.com/paper/2505.16376/) (2025)
- [Skeleton-based Human Action Recognition via Convolutional Neural Networks (CNN)](https://mlsanity.com/paper/2301.13360/) (2023)
- [What's in the Flow? Exploiting Temporal Motion Cues for Unsupervised Generic Event Boundary Detection](https://mlsanity.com/paper/2404.18935/) (2024)
- [Context-aware Proposal Network for Temporal Action Detection](https://mlsanity.com/paper/2206.09082/) (2022)
- [Proposal-based Temporal Action Localization with Point-level Supervision](https://mlsanity.com/paper/2310.05511/) (2023)
- [FMI-TAL: Few-shot Multiple Instances Temporal Action Localization by Probability Distribution Learning and Interval Cluster Refinement](https://mlsanity.com/paper/2408.13765/) (2024)

## Abstract (arXiv)

This technical report presents our solution for temporal action detection task in AcitivityNet Challenge 2021. The purpose of this task is to locate and identify actions of interest in long untrimmed videos. The crucial challenge of the task comes from that the temporal duration of action varies dramatically, and the target actions are typically embedded in a background of irrelevant activities. Our solution builds on BMN, and mainly contains three steps: 1) action classification and feature encoding by Slowfast, CSN and ViViT; 2) proposal generation. We improve BMN by embedding the proposed Proposal Relation Network (PRN), by which we can generate proposals of high quality; 3) action detection. We calculate the detection results by assigning the proposals with corresponding classification results. Finally, we ensemble the results under different settings and achieve 44.7% on the test set, which improves the champion result in ActivityNet 2020 by 1.9% in terms of average mAP.