# Post-Processing Temporal Action Detection

arXiv: https://arxiv.org/abs/2211.14924 · HTML: https://mlsanity.com/paper/2211.14924/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 42 references that are in the map.
- First posted on arXiv 27 Nov 2022.

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Temporal Action Detection with Structured Segment Networks](https://mlsanity.com/paper/1704.06228/) (2017)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [The THUMOS Challenge on Action Recognition for Videos "in the Wild"](https://mlsanity.com/paper/1604.06182/) (2016)
- [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019)
- [Rethinking the Faster R-CNN Architecture for Temporal Action Localization](https://mlsanity.com/paper/1804.07667/) (2018)
- [Distribution-Aware Coordinate Representation for Human Pose Estimation](https://mlsanity.com/paper/1910.06278/) (2019)
- [R-C3D: Region Convolutional 3D Network for Temporal Activity Detection](https://mlsanity.com/paper/1703.07814/) (2017)
- [Learning Salient Boundary Feature for Anchor-free Temporal Action Localization](https://mlsanity.com/paper/2103.13137/) (2021)
- [Gaussian Temporal Awareness Networks for Action Localization](https://mlsanity.com/paper/1909.03877/) (2019)
- [Relaxed Transformer Decoders for Direct Action Proposal Generation](https://mlsanity.com/paper/2102.01894/) (2021)
- [TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals](https://mlsanity.com/paper/1703.06189/) (2017)
- [Temporal Context Aggregation Network for Temporal Action Proposal Refinement](https://mlsanity.com/paper/2103.13141/) (2021)
- [Video Self-Stitching Graph Network for Temporal Action Localization](https://mlsanity.com/paper/2011.14598/) (2020)
- [Diagnosing Error in Temporal Action Detectors](https://mlsanity.com/paper/1807.10706/) (2018)
- [CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning](https://mlsanity.com/paper/2103.16392/) (2021)
- [BSN++: Complementary Boundary Regressor with Scale-Balanced Relation Modeling for Temporal Action Proposal Generation](https://mlsanity.com/paper/2009.07641/) (2020)
- [Zero-Shot Temporal Action Detection via Vision-Language Prompting](https://mlsanity.com/paper/2207.08184/) (2022)
- [Multi-shot Temporal Event Localization: a Benchmark](https://mlsanity.com/paper/2012.09434/) (2020)
- [ReAct: Temporal Action Detection with Relational Queries](https://mlsanity.com/paper/2207.07097/) (2022)
- [Boundary-sensitive Pre-training for Temporal Localization in Videos](https://mlsanity.com/paper/2011.10830/) (2020)
- [ASM-Loc: Action-aware Segment Modeling for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2203.15187/) (2022)
- [Self-Supervised Learning for Semi-Supervised Temporal Action Proposal](https://mlsanity.com/paper/2104.03214/) (2021)
- [Class Semantics-based Attention for Action Detection](https://mlsanity.com/paper/2109.02613/) (2021)
- [DCAN: Improving Temporal Action Detection via Dual Context Aggregation](https://mlsanity.com/paper/2112.03612/) (2021)

## Built on by (most cited first, 6 of 6)

- [TemporalMaxer: Maximize Temporal Context with only Max Pooling for Temporal Action Localization](https://mlsanity.com/paper/2303.09055/) (2023)
- [DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion](https://mlsanity.com/paper/2303.14863/) (2023)
- [Fine-grained Dynamic Network for Generic Event Boundary Detection](https://mlsanity.com/paper/2407.04274/) (2024)
- [SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spotting](https://mlsanity.com/paper/2407.20799/) (2024)
- [Boundary Discretization and Reliable Classification Network for Temporal Action Detection](https://mlsanity.com/paper/2310.06403/) (2023)
- [Multi-Resolution Audio-Visual Feature Fusion for Temporal Action Localization](https://mlsanity.com/paper/2310.03456/) (2023)

## Abstract (arXiv)

Existing Temporal Action Detection (TAD) methods typically take a pre-processing step in converting an input varying-length video into a fixed-length snippet representation sequence, before temporal boundary estimation and action classification. This pre-processing step would temporally downsample the video, reducing the inference resolution and hampering the detection performance in the original temporal resolution. In essence, this is due to a temporal quantization error introduced during the resolution downsampling and recovery. This could negatively impact the TAD performance, but is largely ignored by existing methods. To address this problem, in this work we introduce a novel model-agnostic post-processing method without model redesign and retraining. Specifically, we model the start and end points of action instances with a Gaussian distribution for enabling temporal boundary inference at a sub-snippet level. We further introduce an efficient Taylor-expansion based approximation, dubbed as Gaussian Approximated Post-processing (GAP). Extensive experiments demonstrate that our GAP can consistently improve a wide variety of pre-trained off-the-shelf TAD models on the challenging ActivityNet (+0.2% -0.7% in average mAP) and THUMOS (+0.2% -0.5% in average mAP) benchmarks. Such performance gains are already significant and highly comparable to those achieved by novel model designs. Also, GAP can be integrated with model training for further performance gain. Importantly, GAP enables lower temporal resolutions for more efficient inference, facilitating low-resource applications. The code will be available in https://github.com/sauradip/GAP