# Grouped Spatial-Temporal Aggregation for Efficient Action Recognition

arXiv: https://arxiv.org/abs/1909.13130 · HTML: https://mlsanity.com/paper/1909.13130/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2019.

## Built on by (most cited first, 30 of 55)

- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [UniFormer: Unifying Convolution and Self-attention for Visual Recognition](https://mlsanity.com/paper/2201.09450/) (2022)
- [ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning](https://mlsanity.com/paper/2206.13559/) (2022)
- [UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning](https://mlsanity.com/paper/2201.04676/) (2022)
- [Temporal Query Networks for Fine-grained Video Understanding](https://mlsanity.com/paper/2104.09496/) (2021)
- [ACTION-Net: Multipath Excitation for Action Recognition](https://mlsanity.com/paper/2103.07372/) (2021)
- [MGSampler: An Explainable Sampling Strategy for Video Action Recognition](https://mlsanity.com/paper/2104.09952/) (2021)
- [SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training](https://mlsanity.com/paper/2307.08476/) (2023)
- [AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognition](https://mlsanity.com/paper/2112.14238/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [Learning Self-Similarity in Space and Time as Generalized Motion for Video Action Recognition](https://mlsanity.com/paper/2102.07092/) (2021)
- [Large-scale Robustness Analysis of Video Action Recognition Models](https://mlsanity.com/paper/2207.01398/) (2022)
- [Progressive Attention on Multi-Level Dense Difference Maps for Generic Event Boundary Detection](https://mlsanity.com/paper/2112.04771/) (2021)
- [Stand-Alone Inter-Frame Attention in Video Models](https://mlsanity.com/paper/2206.06931/) (2022)
- [Group Contextualization for Video Recognition](https://mlsanity.com/paper/2203.09694/) (2022)
- [AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognition](https://mlsanity.com/paper/2209.13465/) (2022)
- [Dynamic Spatio-Temporal Specialization Learning for Fine-Grained Action Recognition](https://mlsanity.com/paper/2209.01425/) (2022)
- [DeepPerform: An Efficient Approach for Performance Testing of Resource-Constrained Neural Networks](https://mlsanity.com/paper/2210.05370/) (2022)
- [EAN: Event Adaptive Network for Enhanced Action Recognition](https://mlsanity.com/paper/2107.10771/) (2021)
- [Gate-Shift-Fuse for Video Action Recognition](https://mlsanity.com/paper/2203.08897/) (2022)
- [MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurations](https://mlsanity.com/paper/2105.07085/) (2021)
- [Attention in Attention: Modeling Context Correlation for Efficient Video Classification](https://mlsanity.com/paper/2204.09303/) (2022)
- [What Can Simple Arithmetic Operations Do for Temporal Modeling?](https://mlsanity.com/paper/2307.08908/) (2023)
- [Beyond Short Clips: End-to-End Video-Level Learning with Collaborative Memories](https://mlsanity.com/paper/2104.01198/) (2021)
- [PGT: A Progressive Method for Training Models on Long Videos](https://mlsanity.com/paper/2103.11313/) (2021)
- [Dynamic Temporal Filtering in Video Models](https://mlsanity.com/paper/2211.08252/) (2022)
- [Look, Listen, and Attack: Backdoor Attacks Against Video Action Recognition](https://mlsanity.com/paper/2301.00986/) (2023)
- [Spatiotemporal Deformable Scene Graphs for Complex Activity Detection](https://mlsanity.com/paper/2104.08194/) (2021)
- [Optimization Planning for 3D ConvNets](https://mlsanity.com/paper/2201.04021/) (2022)
- [Boosting Video Representation Learning with Multi-Faceted Integration](https://mlsanity.com/paper/2201.04023/) (2022)

## Abstract (arXiv)

Temporal reasoning is an important aspect of video analysis. 3D CNN shows good performance by exploring spatial-temporal features jointly in an unconstrained way, but it also increases the computational cost a lot. Previous works try to reduce the complexity by decoupling the spatial and temporal filters. In this paper, we propose a novel decomposition method that decomposes the feature channels into spatial and temporal groups in parallel. This decomposition can make two groups focus on static and dynamic cues separately. We call this grouped spatial-temporal aggregation (GST). This decomposition is more parameter-efficient and enables us to quantitatively analyze the contributions of spatial and temporal features in different layers. We verify our model on several action recognition tasks that require temporal reasoning and show its effectiveness.