# Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks

arXiv: https://arxiv.org/abs/1603.07772 · HTML: https://mlsanity.com/paper/1603.07772/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2016.

## Built on by (most cited first, 30 of 63)

- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis](https://mlsanity.com/paper/2107.00285/) (2021)
- [SkateFormer: Skeletal-Temporal Transformer for Human Action Recognition](https://mlsanity.com/paper/2403.09508/) (2024)
- [SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-training](https://mlsanity.com/paper/2307.08476/) (2023)
- [Signature Methods in Machine Learning](https://mlsanity.com/paper/2206.14674/) (2022)
- [IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognition](https://mlsanity.com/paper/2207.12100/) (2022)
- [Hierarchically Self-Supervised Transformer for Human Skeleton Representation Learning](https://mlsanity.com/paper/2207.09644/) (2022)
- [Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognition](https://mlsanity.com/paper/2112.04178/) (2021)
- [A Hierarchical Spatio-Temporal Graph Convolutional Neural Network for Anomaly Detection in Videos](https://mlsanity.com/paper/2112.04294/) (2021)
- [GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition](https://mlsanity.com/paper/2111.13089/) (2021)
- [DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition](https://mlsanity.com/paper/2210.05895/) (2022)
- [Learning Multi-Granular Spatio-Temporal Graph Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2108.04536/) (2021)
- [AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection](https://mlsanity.com/paper/2304.06116/) (2023)
- [Interactive Spatiotemporal Token Attention Network for Skeleton-based General Interactive Action Recognition](https://mlsanity.com/paper/2307.07469/) (2023)
- [Shifted Chunk Transformer for Spatio-Temporal Representational Learning](https://mlsanity.com/paper/2108.11575/) (2021)
- [Learning Connectivity with Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/2112.03328/) (2021)
- [Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2111.11051/) (2021)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [Focal and Global Spatial-Temporal Transformer for Skeleton-based Action Recognition](https://mlsanity.com/paper/2210.02693/) (2022)
- [Coarse Temporal Attention Network (CTA-Net) for Driver's Activity Recognition](https://mlsanity.com/paper/2101.06636/) (2021)
- [Adaptive Hyper-Graph Convolution Network for Skeleton-based Human Action Recognition with Virtual Connections](https://mlsanity.com/paper/2411.14796/) (2024)
- [Motion Matters: Motion-guided Modulation Network for Skeleton-based Micro-Action Recognition](https://mlsanity.com/paper/2507.21977/) (2025)
- [A Two-stream Neural Network for Pose-based Hand Gesture Recognition](https://mlsanity.com/paper/2101.08926/) (2021)
- [Bridging the Skeleton-Text Modality Gap: Diffusion-Powered Modality Alignment for Zero-shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2411.10745/) (2024)
- [Identity-free Artificial Emotional Intelligence via Micro-Gesture Understanding](https://mlsanity.com/paper/2405.13206/) (2024)
- [Video-based Human Action Recognition using Deep Learning: A Review](https://mlsanity.com/paper/2208.03775/) (2022)
- [Skeleton Sequence and RGB Frame Based Multi-Modality Feature Fusion Network for Action Recognition](https://mlsanity.com/paper/2202.11374/) (2022)
- [3D Skeleton-Based Action Recognition: A Review](https://mlsanity.com/paper/2506.00915/) (2025)
- [Domain and View-point Agnostic Hand Action Recognition](https://mlsanity.com/paper/2103.02303/) (2021)
- [Efficient Multiscale Multimodal Bottleneck Transformer for Audio-Video Classification](https://mlsanity.com/paper/2401.04023/) (2024)

## Abstract (arXiv)

Skeleton based action recognition distinguishes human actions using the trajectories of skeleton joints, which provide a very good representation for describing actions. Considering that recurrent neural networks (RNNs) with Long Short-Term Memory (LSTM) can learn feature representations and model long-term temporal dependencies automatically, we propose an end-to-end fully connected deep LSTM network for skeleton based action recognition. Inspired by the observation that the co-occurrences of the joints intrinsically characterize human actions, we take the skeleton as the input at each time slot and introduce a novel regularization scheme to learn the co-occurrence features of skeleton joints. To train the deep LSTM network effectively, we propose a new dropout algorithm which simultaneously operates on the gates, cells, and output responses of the LSTM neurons. Experimental results on three human action recognition datasets consistently demonstrate the effectiveness of the proposed model.