# ACTION-Net: Multipath Excitation for Action Recognition

arXiv: https://arxiv.org/abs/2103.07372 · HTML: https://mlsanity.com/paper/2103.07372/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 51 references that are in the map.
- First posted on arXiv 11 Mar 2021.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Moments in Time Dataset: one million videos for event understanding](https://mlsanity.com/paper/1801.03150/) (2018)
- [TEA: Temporal Excitation and Aggregation for Action Recognition](https://mlsanity.com/paper/2004.01398/) (2020)
- [Temporal Pyramid Network for Action Recognition](https://mlsanity.com/paper/2004.03548/) (2020)
- [STM: SpatioTemporal and Motion Encoding for Action Recognition](https://mlsanity.com/paper/1908.02486/) (2019)
- [TAM: Temporal Adaptive Module for Video Recognition](https://mlsanity.com/paper/2005.06803/) (2020)
- [A2-Nets: Double Attention Networks](https://mlsanity.com/paper/1810.11579/) (2018)
- [ECO: Efficient Convolutional Network for Online Video Understanding](https://mlsanity.com/paper/1804.09066/) (2018)
- [Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networks](https://mlsanity.com/paper/1912.09930/) (2019)
- [Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networks](https://mlsanity.com/paper/1912.06971/) (2019)
- [Involution: Inverting the Inherence of Convolution for Visual Recognition](https://mlsanity.com/paper/2103.06255/) (2021)

## Built on by (most cited first, 30 of 63)

- [Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips](https://mlsanity.com/paper/2404.03663/) (2024)
- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training](https://mlsanity.com/paper/2411.16061/) (2024)
- [HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors](https://mlsanity.com/paper/2211.09648/) (2022)
- [STAR-Transformer: A Spatio-temporal Cross Attention Transformer for Human Action Recognition](https://mlsanity.com/paper/2210.07503/) (2022)
- [SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training](https://mlsanity.com/paper/2307.08476/) (2023)
- [Video Test-Time Adaptation for Action Recognition](https://mlsanity.com/paper/2211.15393/) (2022)
- [ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and More](https://mlsanity.com/paper/2403.12534/) (2024)
- [DirecFormer: A Directed Attention in Transformer Approach to Robust Action Recognition](https://mlsanity.com/paper/2203.10233/) (2022)
- [Causal Reasoning Meets Visual Representation Learning: A Prospective Study](https://mlsanity.com/paper/2204.12037/) (2022)
- [Cross-Modal Learning with 3D Deformable Attention for Action Recognition](https://mlsanity.com/paper/2212.05638/) (2022)
- [SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition](https://mlsanity.com/paper/2308.04369/) (2023)
- [Stand-Alone Inter-Frame Attention in Video Models](https://mlsanity.com/paper/2206.06931/) (2022)
- [Spatiotemporal Self-attention Modeling with Temporal Patch Shift for Action Recognition](https://mlsanity.com/paper/2207.13259/) (2022)
- [Inherent Redundancy in Spiking Neural Networks](https://mlsanity.com/paper/2308.08227/) (2023)
- [Event Stream based Human Action Recognition: A High-Definition Benchmark Dataset and Algorithms](https://mlsanity.com/paper/2408.09764/) (2024)
- [SpikMamba: When SNN meets Mamba in Event-based Human Action Recognition](https://mlsanity.com/paper/2410.16746/) (2024)
- [Motion-driven Visual Tempo Learning for Video-based Action Recognition](https://mlsanity.com/paper/2202.12116/) (2022)
- [MLP-3D: A MLP-like 3D Architecture with Grouped Time Mixing](https://mlsanity.com/paper/2206.06292/) (2022)
- [Multi-Task and Multi-Modal Learning for RGB Dynamic Gesture Recognition](https://mlsanity.com/paper/2110.15639/) (2021)
- [Uncertainty-Driven Action Quality Assessment](https://mlsanity.com/paper/2207.14513/) (2022)
- [Dynamic Temporal Filtering in Video Models](https://mlsanity.com/paper/2211.08252/) (2022)
- [Semantic-Aware Frame-Event Fusion based Pattern Recognition via Large Vision-Language Models](https://mlsanity.com/paper/2311.18592/) (2023)
- [CycDA: Unsupervised Cycle Domain Adaptation from Image to Video](https://mlsanity.com/paper/2203.16244/) (2022)
- [MODEM: A Morton-Order Degradation Estimation Mechanism for Adverse Weather Image Recovery](https://mlsanity.com/paper/2505.17581/) (2025)
- [Inter-intra Variant Dual Representations forSelf-supervised Video Recognition](https://mlsanity.com/paper/2107.01194/) (2021)
- [TEAM-Net: Multi-modal Learning for Video Action Recognition with Partial Decoding](https://mlsanity.com/paper/2110.08814/) (2021)
- [Maximizing Spatio-Temporal Entropy of Deep 3D CNNs for Efficient Video Recognition](https://mlsanity.com/paper/2303.02693/) (2023)
- [Unleashing the Power of CNN and Transformer for Balanced RGB-Event Video Recognition](https://mlsanity.com/paper/2312.11128/) (2023)
- [RNNs, CNNs and Transformers in Human Action Recognition: A Survey and a Hybrid Model](https://mlsanity.com/paper/2407.06162/) (2024)

## Abstract (arXiv)

Spatial-temporal, channel-wise, and motion patterns are three complementary and crucial types of information for video action recognition. Conventional 2D CNNs are computationally cheap but cannot catch temporal relationships; 3D CNNs can achieve good performance but are computationally intensive. In this work, we tackle this dilemma by designing a generic and effective module that can be embedded into 2D CNNs. To this end, we propose a spAtio-temporal, Channel and moTion excitatION (ACTION) module consisting of three paths: Spatio-Temporal Excitation (STE) path, Channel Excitation (CE) path, and Motion Excitation (ME) path. The STE path employs one channel 3D convolution to characterize spatio-temporal representation. The CE path adaptively recalibrates channel-wise feature responses by explicitly modeling interdependencies between channels in terms of the temporal aspect. The ME path calculates feature-level temporal differences, which is then utilized to excite motion-sensitive channels. We equip 2D CNNs with the proposed ACTION module to form a simple yet effective ACTION-Net with very limited extra computational cost. ACTION-Net is demonstrated by consistently outperforming 2D CNN counterparts on three backbones (i.e., ResNet-50, MobileNet V2 and BNInception) employing three datasets (i.e., Something-Something V2, Jester, and EgoGesture). Codes are available at urlhttps://github.com/V-Sense/ACTION-Net.