# Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates

arXiv: https://arxiv.org/abs/1706.08276 · HTML: https://mlsanity.com/paper/1706.08276/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2017.

## Built on by (most cited first, 30 of 54)

- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [Hierarchically Decomposed Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2208.10741/) (2022)
- [Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learning](https://mlsanity.com/paper/2108.01959/) (2021)
- [Deep Gait Recognition: A Survey](https://mlsanity.com/paper/2102.09546/) (2021)
- [Masked Motion Predictors are Strong 3D Action Representation Learners](https://mlsanity.com/paper/2308.07092/) (2023)
- [Trear: Transformer-based RGB-D Egocentric Action Recognition](https://mlsanity.com/paper/2101.03904/) (2021)
- [Signature Methods in Machine Learning](https://mlsanity.com/paper/2206.14674/) (2022)
- [GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition](https://mlsanity.com/paper/2111.13089/) (2021)
- [Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2110.05092/) (2021)
- [Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semi-Supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2302.02316/) (2023)
- [Self-Supervised 3D Action Representation Learning with Skeleton Cloud Colorization](https://mlsanity.com/paper/2304.08799/) (2023)
- [One-shot action recognition in challenging therapy scenarios](https://mlsanity.com/paper/2102.08997/) (2021)
- [Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition](https://mlsanity.com/paper/2110.14994/) (2021)
- [Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions](https://mlsanity.com/paper/2202.11423/) (2022)
- [Concept Graph Neural Networks for Surgical Video Understanding](https://mlsanity.com/paper/2202.13402/) (2022)
- [Leveraging Spatio-Temporal Dependency for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2212.04761/) (2022)
- [Attention-Propagation Network for Egocentric Heatmap to 3D Pose Lifting](https://mlsanity.com/paper/2402.18330/) (2024)
- [One-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching](https://mlsanity.com/paper/2307.07286/) (2023)
- [Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition](https://mlsanity.com/paper/2110.13008/) (2021)
- [Pyramid Self-attention Polymerization Learning for Semi-supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2302.02327/) (2023)
- [Adaptive Hyper-Graph Convolution Network for Skeleton-based Human Action Recognition with Virtual Connections](https://mlsanity.com/paper/2411.14796/) (2024)
- [CapsField: Light Field-based Face and Expression Recognition in the Wild using Capsule Routing](https://mlsanity.com/paper/2101.03503/) (2021)
- [Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2209.10073/) (2022)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)
- [Two-person Graph Convolutional Network for Skeleton-based Human Interaction Recognition](https://mlsanity.com/paper/2208.06174/) (2022)
- [Towards Physical World Backdoor Attacks against Skeleton Action Recognition](https://mlsanity.com/paper/2408.08671/) (2024)
- [Associated Spatio-Temporal Capsule Network for Gait Recognition](https://mlsanity.com/paper/2101.02458/) (2021)
- [3D Skeleton-Based Action Recognition: A Review](https://mlsanity.com/paper/2506.00915/) (2025)
- [MK-SGN: A Spiking Graph Convolutional Network with Multimodal Fusion and Knowledge Distillation for Skeleton-based Action Recognition](https://mlsanity.com/paper/2404.10210/) (2024)
- [MoFM: A Large-Scale Human Motion Foundation Model](https://mlsanity.com/paper/2502.05432/) (2025)

## Abstract (arXiv)

Skeleton-based human action recognition has attracted a lot of research attention during the past few years. Recent works attempted to utilize recurrent neural networks to model the temporal dependencies between the 3D positional configurations of human body joints for better analysis of human activities in the skeletal data. The proposed work extends this idea to spatial domain as well as temporal domain to better analyze the hidden sources of action-related information within the human skeleton sequences in both of these domains simultaneously. Based on the pictorial structure of Kinect's skeletal data, an effective tree-structure based traversal framework is also proposed. In order to deal with the noise in the skeletal data, a new gating mechanism within LSTM module is introduced, with which the network can learn the reliability of the sequential data and accordingly adjust the effect of the input data on the updating procedure of the long-term context representation stored in the unit's memory cell. Moreover, we introduce a novel multi-modal feature fusion strategy within the LSTM unit in this paper. The comprehensive experimental results on seven challenging benchmark datasets for human action recognition demonstrate the effectiveness of the proposed method.