# Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition

arXiv: https://arxiv.org/abs/2110.13008 · HTML: https://mlsanity.com/paper/2110.13008/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 30 of 32)

- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition](https://mlsanity.com/paper/1607.07043/) (2016)
- [Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation](https://mlsanity.com/paper/1804.06055/) (2018)
- [A Primer on the Signature Method in Machine Learning](https://mlsanity.com/paper/1603.03788/) (2016)
- [Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition](https://mlsanity.com/paper/2010.09978/) (2020)
- [Kernels for sequentially ordered data](https://mlsanity.com/paper/1601.08169/) (2016)
- [Deep Signature Transforms](https://mlsanity.com/paper/1905.08494/) (2019)
- [Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks](https://mlsanity.com/paper/1707.05740/) (2017)
- [P-CNN: Pose-based CNN Features for Action Recognition](https://mlsanity.com/paper/1506.03607/) (2015)
- [Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates](https://mlsanity.com/paper/1706.08276/) (2017)
- [Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks](https://mlsanity.com/paper/1704.02581/) (2017)
- [Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU](https://mlsanity.com/paper/2001.00706/) (2020)
- [SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition](https://mlsanity.com/paper/1907.13025/) (2019)
- [The iisignature library: efficient calculation of iterated-integral signatures and log signatures](https://mlsanity.com/paper/1802.08252/) (2018)
- [Richly Activated Graph Convolutional Network for Robust Skeleton-based Action Recognition](https://mlsanity.com/paper/2008.03791/) (2020)
- [A Comparative Review of Recent Kinect-based Action Recognition Algorithms](https://mlsanity.com/paper/1906.09955/) (2019)
- [Skeleton-Based Online Action Prediction Using Scale Selection Network](https://mlsanity.com/paper/1902.03084/) (2019)
- [Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition](https://mlsanity.com/paper/1610.02616/) (2016)
- [Sparse arrays of signatures for online character recognition](https://mlsanity.com/paper/1308.0371/) (2013)
- [A signature-based machine learning model for bipolar disorder and borderline personality disorder](https://mlsanity.com/paper/1707.07124/) (2017)
- [DeepWriterID: An End-to-end Online Text-independent Writer Identification System](https://mlsanity.com/paper/1508.04945/) (2015)
- [Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module](https://mlsanity.com/paper/1811.07081/) (2018)
- [DropSample: A New Training Method to Enhance Deep Convolutional Neural Networks for Large-Scale Unconstrained Handwritten Chinese Character Recognition](https://mlsanity.com/paper/1505.05354/) (2015)
- [RNN Fisher Vectors for Action Recognition and Image Annotation](https://mlsanity.com/paper/1512.03958/) (2015)
- [Online Signature Verification using Recurrent Neural Network and Length-normalized Path Signature](https://mlsanity.com/paper/1705.06849/) (2017)

## Built on by (most cited first, 11 of 11)

- [Sig-Wasserstein GANs for Time Series Generation](https://mlsanity.com/paper/2111.01207/) (2021)
- [3D Skeleton-Based Action Recognition: A Review](https://mlsanity.com/paper/2506.00915/) (2025)
- [GCN-DevLSTM: Path Development for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2403.15212/) (2024)
- [Using joint angles based on the international biomechanical standards for human action recognition and related tasks](https://mlsanity.com/paper/2406.17443/) (2024)
- [Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter](https://mlsanity.com/paper/2508.16939/) (2025)
- [A Neural RDE-based model for solving path-dependent PDEs](https://mlsanity.com/paper/2306.01123/) (2023)
- [Sig-Networks Toolkit: Signature Networks for Longitudinal Language Modelling](https://mlsanity.com/paper/2312.03523/) (2023)
- [SHARDeg: A Benchmark for Skeletal Human Action Recognition in Degraded Scenarios](https://mlsanity.com/paper/2505.18048/) (2025)
- [Learning non-Markovian Dynamical Systems with Signature-based Encoders](https://mlsanity.com/paper/2509.12022/) (2025)
- [pathsig: A GPU-Accelerated Library for Truncated and Projected Path Signatures](https://mlsanity.com/paper/2602.24066/) (2026)
- [LogSig-SSM: Time-Series Modelling with Multi-Scale Log-Signature Compression for State-Space Models](https://mlsanity.com/paper/2610.05051/) (2026)

## Abstract (arXiv)

This paper contributes to the challenge of skeleton-based human action recognition in videos. The key step is to develop a generic network architecture to extract discriminative features for the spatio-temporal skeleton data. In this paper, we propose a novel module, namely Logsig-RNN, which is the combination of the log-signature layer and recurrent type neural networks (RNNs). The former one comes from the mathematically principled technology of signatures and log-signatures as representations for streamed data, which can manage high sample rate streams, non-uniform sampling and time series of variable length. It serves as an enhancement of the recurrent layer, which can be conveniently plugged into neural networks. Besides we propose two path transformation layers to significantly reduce path dimension while retaining the essential information fed into the Logsig-RNN module. Finally, numerical results demonstrate that replacing the RNN module by the Logsig-RNN module in SOTA networks consistently improves the performance on both Chalearn gesture data and NTU RGB+D 120 action data in terms of accuracy and robustness. In particular, we achieve the state-of-the-art accuracy on Chalearn2013 gesture data by combining simple path transformation layers with the Logsig-RNN. Codes are available at https://github.com/steveliao93/GCN_LogsigRNN.