# Context-aware Proposal Network for Temporal Action Detection

arXiv: https://arxiv.org/abs/2206.09082 · HTML: https://mlsanity.com/paper/2206.09082/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 34 references that are in the map.
- First posted on arXiv 18 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [A Short Note on the Kinetics-700 Human Action Dataset](https://mlsanity.com/paper/1907.06987/) (2019)
- [Video Classification with Channel-Separated Convolutional Networks](https://mlsanity.com/paper/1904.02811/) (2019)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [Temporal Pyramid Network for Action Recognition](https://mlsanity.com/paper/2004.03548/) (2020)
- [Temporal Context Aggregation Network for Temporal Action Proposal Refinement](https://mlsanity.com/paper/2103.13141/) (2021)
- [OadTR: Online Action Detection with Transformers](https://mlsanity.com/paper/2106.11149/) (2021)
- [TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Tasks](https://mlsanity.com/paper/2011.11479/) (2020)
- [Multi-shot Temporal Event Localization: a Benchmark](https://mlsanity.com/paper/2012.09434/) (2020)
- [Hybrid Relation Guided Set Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2204.13423/) (2022)
- [Self-Supervised Learning for Semi-Supervised Temporal Action Proposal](https://mlsanity.com/paper/2104.03214/) (2021)
- [TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection](https://mlsanity.com/paper/1905.13417/) (2019)
- [RCL: Recurrent Continuous Localization for Temporal Action Detection](https://mlsanity.com/paper/2203.07112/) (2022)
- [Self-supervised Motion Learning from Static Images](https://mlsanity.com/paper/2104.00240/) (2021)
- [Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization](https://mlsanity.com/paper/2103.15233/) (2021)
- [Proposal Relation Network for Temporal Action Detection](https://mlsanity.com/paper/2106.11812/) (2021)
- [CBR-Net: Cascade Boundary Refinement Network for Action Detection: Submission to ActivityNet Challenge 2020 (Task 1)](https://mlsanity.com/paper/2006.07526/) (2020)
- [Multi-Level Temporal Pyramid Network for Action Detection](https://mlsanity.com/paper/2008.03270/) (2020)
- [Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling](https://mlsanity.com/paper/2106.11811/) (2021)

## Built on by (most cited first, 3 of 3)

- [MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition](https://mlsanity.com/paper/2304.00946/) (2023)
- [HR-Pro: Point-supervised Temporal Action Localization via Hierarchical Reliability Propagation](https://mlsanity.com/paper/2308.12608/) (2023)
- [Boosting Point-supervised Temporal Action Localization via Text Refinement and Alignment](https://mlsanity.com/paper/2602.01257/) (2026)

## Abstract (arXiv)

This technical report presents our first place winning solution for temporal action detection task in CVPR-2022 AcitivityNet Challenge. The task aims to localize temporal boundaries of action instances with specific classes in long untrimmed videos. Recent mainstream attempts are based on dense boundary matchings and enumerate all possible combinations to produce proposals. We argue that the generated proposals contain rich contextual information, which may benefits detection confidence prediction. To this end, our method mainly consists of the following three steps: 1) action classification and feature extraction by Slowfast, CSN, TimeSformer, TSP, I3D-flow, VGGish-audio, TPN and ViViT; 2) proposal generation. Our proposed Context-aware Proposal Network (CPN) builds on top of BMN, GTAD and PRN to aggregate contextual information by randomly masking some proposal features. 3) action detection. The final detection prediction is calculated by assigning the proposals with corresponding video-level classifcation results. Finally, we ensemble the results under different feature combination settings and achieve 45.8% performance on the test set, which improves the champion result in CVPR-2021 ActivityNet Challenge by 1.1% in terms of average mAP.