# InfoGCN++: Learning Representation by Predicting the Future for Online Human Skeleton-based Action Recognition

arXiv: https://arxiv.org/abs/2310.10547 · HTML: https://mlsanity.com/paper/2310.10547/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 64 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time](https://mlsanity.com/paper/2211.03375/) (2022)
- [Video Representation Learning by Dense Predictive Coding](https://mlsanity.com/paper/1909.04656/) (2019)
- [Memory-augmented Dense Predictive Coding for Video Representation Learning](https://mlsanity.com/paper/2008.01065/) (2020)
- [Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition](https://mlsanity.com/paper/1904.01189/) (2019)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [OadTR: Online Action Detection with Transformers](https://mlsanity.com/paper/2106.11149/) (2021)
- [Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition](https://mlsanity.com/paper/2007.14690/) (2020)
- [Temporal Recurrent Networks for Online Action Detection](https://mlsanity.com/paper/1811.07391/) (2018)
- [Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2206.13028/) (2022)
- [Online Action Detection](https://mlsanity.com/paper/1604.06506/) (2016)
- [Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equation](https://mlsanity.com/paper/2010.08188/) (2020)
- [3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition](https://mlsanity.com/paper/2303.14474/) (2023)
- [IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognition](https://mlsanity.com/paper/2207.12100/) (2022)
- [Focal and Global Spatial-Temporal Transformer for Skeleton-based Action Recognition](https://mlsanity.com/paper/2210.02693/) (2022)
- [Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition](https://mlsanity.com/paper/1802.09834/) (2018)

## Built on by (most cited first, 11 of 11)

- [Deep Learning Approaches for Seizure Video Analysis: A Review](https://mlsanity.com/paper/2312.10930/) (2023)
- [Generative Hierarchical Temporal Transformer for Hand Pose and Action Modeling](https://mlsanity.com/paper/2311.17366/) (2023)
- [DuoCLR: Dual-Surrogate Contrastive Learning for Skeleton-based Human Action Segmentation](https://mlsanity.com/paper/2509.05543/) (2025)
- [MistExit: Learning to Exit for Early Mistake Detection in Procedural Videos](https://mlsanity.com/paper/2603.14252/) (2026)
- [EgoLoc: A Generalizable Solution for Temporal Interaction Localization in Egocentric Videos](https://mlsanity.com/paper/2508.12349/) (2025)
- [Heatmap Pooling Network for Action Recognition from RGB Videos](https://mlsanity.com/paper/2512.03837/) (2025)
- [CGTGait: Collaborative Graph and Transformer for Gait Emotion Recognition](https://mlsanity.com/paper/2509.16623/) (2025)
- [TSkel-Mamba: Temporal Dynamic Modeling via State Space Model for Human Skeleton-based Action Recognition](https://mlsanity.com/paper/2512.11503/) (2025)
- [Mining Multi-Modality Spatio-Temporal Cues for Video Important Person Identification](https://mlsanity.com/paper/2605.28604/) (2026)
- [Real-time body pose non-verbal communication with a consistency-based reliability measure](https://mlsanity.com/paper/2606.09390/) (2026)
- [Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming](https://mlsanity.com/paper/2608.25495/) (2026)

## Abstract (arXiv)

Skeleton-based action recognition has made significant advancements recently, with models like InfoGCN showcasing remarkable accuracy. However, these models exhibit a key limitation: they necessitate complete action observation prior to classification, which constrains their applicability in real-time situations such as surveillance and robotic systems. To overcome this barrier, we introduce InfoGCN++, an innovative extension of InfoGCN, explicitly developed for online skeleton-based action recognition. InfoGCN++ augments the abilities of the original InfoGCN model by allowing real-time categorization of action types, independent of the observation sequence's length. It transcends conventional approaches by learning from current and anticipated future movements, thereby creating a more thorough representation of the entire sequence. Our approach to prediction is managed as an extrapolation issue, grounded on observed actions. To enable this, InfoGCN++ incorporates Neural Ordinary Differential Equations, a concept that lets it effectively model the continuous evolution of hidden states. Following rigorous evaluations on three skeleton-based action recognition benchmarks, InfoGCN++ demonstrates exceptional performance in online action recognition. It consistently equals or exceeds existing techniques, highlighting its significant potential to reshape the landscape of real-time action recognition applications. Consequently, this work represents a major leap forward from InfoGCN, pushing the limits of what's possible in online, skeleton-based action recognition. The code for InfoGCN++ is publicly available at https://github.com/stnoah1/infogcn2 for further exploration and validation.