# Human Activity Recognition Using Cascaded Dual Attention CNN and Bi-Directional GRU Framework

arXiv: https://arxiv.org/abs/2208.05034 · HTML: https://mlsanity.com/paper/2208.05034/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 114 references that are in the map.
- First posted on arXiv 9 Aug 2022.

## Builds on (most cited first, 30 of 45)

- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [A Short Note about Kinetics-600](https://mlsanity.com/paper/1808.01340/) (2018)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [Multiview Transformers for Video Recognition](https://mlsanity.com/paper/2201.04288/) (2022)
- [STM: SpatioTemporal and Motion Encoding for Action Recognition](https://mlsanity.com/paper/1908.02486/) (2019)
- [MoViNets: Mobile Video Networks for Efficient Video Recognition](https://mlsanity.com/paper/2103.11511/) (2021)
- [Evidential Deep Learning for Open Set Action Recognition](https://mlsanity.com/paper/2107.10161/) (2021)
- [Hierarchical Deep Temporal Models for Group Activity Recognition](https://mlsanity.com/paper/1607.02643/) (2016)
- [Long-term Temporal Convolutions for Action Recognition](https://mlsanity.com/paper/1604.04494/) (2016)
- [Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition](https://mlsanity.com/paper/1711.11152/) (2017)
- [Self-supervised Video Transformer](https://mlsanity.com/paper/2112.01514/) (2021)
- [Learning Spatio-Temporal Representation with Local and Global Diffusion](https://mlsanity.com/paper/1906.05571/) (2019)
- [VideoLSTM Convolves, Attends and Flows for Action Recognition](https://mlsanity.com/paper/1607.01794/) (2016)
- [D3D: Distilled 3D Networks for Video Action Recognition](https://mlsanity.com/paper/1812.08249/) (2018)
- [MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognition](https://mlsanity.com/paper/2108.09322/) (2021)
- [End-to-End Learning of Motion Representation for Video Understanding](https://mlsanity.com/paper/1804.00413/) (2018)
- [Cascaded Human-Object Interaction Recognition](https://mlsanity.com/paper/2003.04262/) (2020)
- [Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition](https://mlsanity.com/paper/1912.08077/) (2019)
- [Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition](https://mlsanity.com/paper/1712.05134/) (2017)
- [Actions ~ Transformations](https://mlsanity.com/paper/1512.00795/) (2015)
- [Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification](https://mlsanity.com/paper/1711.09550/) (2017)
- [StNet: Local and Global Spatial-Temporal Modeling for Action Recognition](https://mlsanity.com/paper/1811.01549/) (2018)

## Built on by (most cited first, 1 of 1)

- [A Bidirectional Siamese Recurrent Neural Network for Accurate Gait Recognition Using Body Landmarks](https://mlsanity.com/paper/2412.03498/) (2024)

## Abstract (arXiv)

Vision-based human activity recognition has emerged as one of the essential research areas in video analytics domain. Over the last decade, numerous advanced deep learning algorithms have been introduced to recognize complex human actions from video streams. These deep learning algorithms have shown impressive performance for the human activity recognition task. However, these newly introduced methods either exclusively focus on model performance or the effectiveness of these models in terms of computational efficiency and robustness, resulting in a biased tradeoff in their proposals to deal with challenging human activity recognition problem. To overcome the limitations of contemporary deep learning models for human activity recognition, this paper presents a computationally efficient yet generic spatial-temporal cascaded framework that exploits the deep discriminative spatial and temporal features for human activity recognition. For efficient representation of human actions, we have proposed an efficient dual attentional convolutional neural network (CNN) architecture that leverages a unified channel-spatial attention mechanism to extract human-centric salient features in video frames. The dual channel-spatial attention layers together with the convolutional layers learn to be more attentive in the spatial receptive fields having objects over the number of feature maps. The extracted discriminative salient features are then forwarded to stacked bi-directional gated recurrent unit (Bi-GRU) for long-term temporal modeling and recognition of human actions using both forward and backward pass gradient learning. Extensive experiments are conducted, where the obtained results show that the proposed framework attains an improvement in execution time up to 167 times in terms of frames per second as compared to most of the contemporary action recognition methods.