# TNTC: two-stream network with transformer-based complementarity for gait-based emotion recognition

arXiv: https://arxiv.org/abs/2110.13708 · HTML: https://mlsanity.com/paper/2110.13708/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits](https://mlsanity.com/paper/1910.12906/) (2019)
- [ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation](https://mlsanity.com/paper/2003.01062/) (2020)
- [Identifying Emotions from Walking using Affective and Deep Features](https://mlsanity.com/paper/1906.11884/) (2019)
- [Emotion Recognition From Gait Analyses: Current Research and Future Directions](https://mlsanity.com/paper/2003.11461/) (2020)
- [Take an Emotion Walk: Perceiving Emotions from Gaits Using Hierarchical Attention Pooling and Affective Mapping](https://mlsanity.com/paper/1911.08708/) (2019)

## Built on by (most cited first, 6 of 6)

- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [Emotion Recognition from Skeleton Data: A Comprehensive Survey](https://mlsanity.com/paper/2507.18026/) (2025)
- [ASD: Towards Attribute Spatial Decomposition for Prior-Free Facial Attribute Recognition](https://mlsanity.com/paper/2210.13716/) (2022)
- [Emotion Loss Attacking: Adversarial Attack Perception for Skeleton based on Multi-dimensional Features](https://mlsanity.com/paper/2406.19815/) (2024)
- [CGTGait: Collaborative Graph and Transformer for Gait Emotion Recognition](https://mlsanity.com/paper/2509.16623/) (2025)
- [Single-Stream Multi-Feature Fusion with Temporal Robustness for Gait Emotion Recognition](https://mlsanity.com/paper/2609.11680/) (2026)

## Abstract (arXiv)

Recognizing the human emotion automatically from visual characteristics plays a vital role in many intelligent applications. Recently, gait-based emotion recognition, especially gait skeletons-based characteristic, has attracted much attention, while many available methods have been proposed gradually. The popular pipeline is to first extract affective features from joint skeletons, and then aggregate the skeleton joint and affective features as the feature vector for classifying the emotion. However, the aggregation procedure of these emerged methods might be rigid, resulting in insufficiently exploiting the complementary relationship between skeleton joint and affective features. Meanwhile, the long range dependencies in both spatial and temporal domains of the gait sequence are scarcely considered. To address these issues, we propose a novel two-stream network with transformer-based complementarity, termed as TNTC. Skeleton joint and affective features are encoded into two individual images as the inputs of two streams, respectively. A new transformer-based complementarity module (TCM) is proposed to bridge the complementarity between two streams hierarchically via capturing long range dependencies. Experimental results demonstrate TNTC outperforms state-of-the-art methods on the latest dataset in terms of accuracy.