# Depth Pooling Based Large-scale 3D Action Recognition with Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1804.01194 · HTML: https://mlsanity.com/paper/1804.01194/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2018.

## Built on by (most cited first, 19 of 19)

- [MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2111.12707/) (2021)
- [Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2103.14304/) (2021)
- [PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences](https://mlsanity.com/paper/2205.13713/) (2022)
- [Augmented Transformer with Adaptive Graph for Temporal Action Proposal Generation](https://mlsanity.com/paper/2103.16024/) (2021)
- [No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfaces](https://mlsanity.com/paper/2203.11113/) (2022)
- [Hourglass Tokenizer for Efficient Transformer-Based 3D Human Pose Estimation](https://mlsanity.com/paper/2311.12028/) (2023)
- [Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformer](https://mlsanity.com/paper/2407.12322/) (2024)
- [Decoupling and Recoupling Spatiotemporal Representation for RGB-D-based Motion Recognition](https://mlsanity.com/paper/2112.09129/) (2021)
- [Real-time 3D human action recognition based on Hyperpoint sequence](https://mlsanity.com/paper/2111.08492/) (2021)
- [A Unified Multimodal De- and Re-coupling Framework for RGB-D Motion Recognition](https://mlsanity.com/paper/2211.09146/) (2022)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)
- [STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action Recognition](https://mlsanity.com/paper/2303.18177/) (2023)
- [Spatial-temporal Transformer-guided Diffusion based Data Augmentation for Efficient Skeleton-based Action Recognition](https://mlsanity.com/paper/2302.13434/) (2023)
- [Double-chain Constraints for 3D Human Pose Estimation in Images and Videos](https://mlsanity.com/paper/2308.05298/) (2023)
- [Transformers in Action Recognition: A Review on Temporal Modeling](https://mlsanity.com/paper/2302.01921/) (2022)
- [Joint Temporal Pooling for Improving Skeleton-based Action Recognition](https://mlsanity.com/paper/2408.09356/) (2024)
- [Content-Distortion High-Order Interaction for Blind Image Quality Assessment](https://mlsanity.com/paper/2504.05076/) (2025)
- [Human Action Recognition Based on Multi-scale Feature Maps from Depth Video Sequences](https://mlsanity.com/paper/2101.07618/) (2021)
- [GATS: Gaussian Aware Temporal Scaling Transformer for Invariant 4D Spatio-Temporal Point Cloud Representation](https://mlsanity.com/paper/2603.16154/) (2026)

## Abstract (arXiv)

This paper proposes three simple, compact yet effective representations of depth sequences, referred to respectively as Dynamic Depth Images (DDI), Dynamic Depth Normal Images (DDNI) and Dynamic Depth Motion Normal Images (DDMNI), for both isolated and continuous action recognition. These dynamic images are constructed from a segmented sequence of depth maps using hierarchical bidirectional rank pooling to effectively capture the spatial-temporal information. Specifically, DDI exploits the dynamics of postures over time and DDNI and DDMNI exploit the 3D structural information captured by depth maps. Upon the proposed representations, a ConvNet based method is developed for action recognition. The image-based representations enable us to fine-tune the existing Convolutional Neural Network (ConvNet) models trained on image data without training a large number of parameters from scratch. The proposed method achieved the state-of-art results on three large datasets, namely, the Large-scale Continuous Gesture Recognition Dataset (means Jaccard index 0.4109), the Large-scale Isolated Gesture Recognition Dataset (59.21%), and the NTU RGB+D Dataset (87.08% cross-subject and 84.22% cross-view) even though only the depth modality was used.