# STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits

arXiv: https://arxiv.org/abs/1910.12906 · HTML: https://mlsanity.com/paper/1910.12906/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 30 of 36)

- [Speech2AffectiveGestures: Synthesizing Co-Speech Gestures with Generative Adversarial Affective Expression Learning](https://mlsanity.com/paper/2108.00262/) (2021)
- [Context De-confounded Emotion Recognition](https://mlsanity.com/paper/2303.11921/) (2023)
- [Robust Emotion Recognition in Context Debiasing](https://mlsanity.com/paper/2403.05963/) (2024)
- [Spatio-Temporal Graph Neural Networks: A Survey](https://mlsanity.com/paper/2301.10569/) (2023)
- [Emotion Recognition from Multiple Modalities: Fundamentals and Methodologies](https://mlsanity.com/paper/2108.10152/) (2021)
- [VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoning](https://mlsanity.com/paper/2404.07078/) (2024)
- [Leveraging Semantic Scene Characteristics and Multi-Stream Convolutional Architectures in a Contextual Approach for Video-Based Visual Emotion Recognition in the Wild](https://mlsanity.com/paper/2105.07484/) (2021)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [DynSTGAT: Dynamic Spatial-Temporal Graph Attention Network for Traffic Signal Control](https://mlsanity.com/paper/2109.05491/) (2021)
- [Privacy-Preserving Video Classification with Convolutional Neural Networks](https://mlsanity.com/paper/2102.03513/) (2021)
- [Computer Vision for Clinical Gait Analysis: A Gait Abnormality Video Dataset](https://mlsanity.com/paper/2407.04190/) (2024)
- [3MASSIV: Multilingual, Multimodal and Multi-Aspect dataset of Social Media Short Videos](https://mlsanity.com/paper/2203.14456/) (2022)
- [TNTC: two-stream network with transformer-based complementarity for gait-based emotion recognition](https://mlsanity.com/paper/2110.13708/) (2021)
- [MSA-GCN:Multiscale Adaptive Graph Convolution Network for Gait Emotion Recognition](https://mlsanity.com/paper/2209.08988/) (2022)
- [Graph Neural Network for spatiotemporal data: methods and applications](https://mlsanity.com/paper/2306.00012/) (2023)
- [Graph Neural Networks in Computer Vision -- Architectures, Datasets and Common Approaches](https://mlsanity.com/paper/2212.10207/) (2022)
- [HighlightMe: Detecting Highlights from Human-Centric Videos](https://mlsanity.com/paper/2110.01774/) (2021)
- [Data Augmentation for Depression Detection Using Skeleton-Based Gait Information](https://mlsanity.com/paper/2201.01115/) (2022)
- [Bodily expressed emotion understanding through integrating Laban movement analysis](https://mlsanity.com/paper/2304.02187/) (2023)
- [EMERSK -- Explainable Multimodal Emotion Recognition with Situational Knowledge](https://mlsanity.com/paper/2306.08657/) (2023)
- [ST-Gait++: Leveraging spatio-temporal convolutions for gait-based emotion recognition on videos](https://mlsanity.com/paper/2405.13903/) (2024)
- [Fuzzy-aware Loss for Source-free Domain Adaptation in Visual Emotion Recognition](https://mlsanity.com/paper/2501.15519/) (2025)
- [Mutilmodal Feature Extraction and Attention-based Fusion for Emotion Estimation in Videos](https://mlsanity.com/paper/2303.10421/) (2023)
- [Emotion Recognition from Skeleton Data: A Comprehensive Survey](https://mlsanity.com/paper/2507.18026/) (2025)
- [PhysiQ: Off-site Quality Assessment of Exercise in Physical Therapy](https://mlsanity.com/paper/2211.08245/) (2022)
- [Using Scene and Semantic Features for Multi-modal Emotion Recognition](https://mlsanity.com/paper/2308.00228/) (2023)
- [VRMN-bD: A Multi-modal Natural Behavior Dataset of Immersive Human Fear Responses in VR Stand-up Interactive Games](https://mlsanity.com/paper/2401.12133/) (2024)
- [Self-supervised Gait-based Emotion Representation Learning from Selective Strongly Augmented Skeleton Sequences](https://mlsanity.com/paper/2405.04900/) (2024)
- [Developing Normative Gait Cycle Parameters for Clinical Analysis Using Human Pose Estimation](https://mlsanity.com/paper/2411.13716/) (2024)
- [Multi-Cue Adaptive Emotion Recognition Network](https://mlsanity.com/paper/2111.02273/) (2021)

## Abstract (arXiv)

We present a novel classifier network called STEP, to classify perceived human emotion from gaits, based on a Spatial Temporal Graph Convolutional Network (ST-GCN) architecture. Given an RGB video of an individual walking, our formulation implicitly exploits the gait features to classify the emotional state of the human into one of four emotions: happy, sad, angry, or neutral. We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE). We incorporate a novel push-pull regularization loss in the CVAE formulation of STEP-Gen to generate realistic gaits and improve the classification accuracy of STEP. We also release a novel dataset (E-Gait), which consists of 2,177 human gaits annotated with perceived emotions along with thousands of synthetic gaits. In practice, STEP can learn the affective features and exhibits classification accuracy of 89% on E-Gait, which is 14 - 30% more accurate over prior methods.