# Skeleton-Guided Spatial-Temporal Feature Learning for Video-Based Visible-Infrared Person Re-Identification

arXiv: https://arxiv.org/abs/2411.11069 · HTML: https://mlsanity.com/paper/2411.11069/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 17 Nov 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2007.09314/) (2020)
- [Cross-modality Person re-identification with Shared-Specific Feature Transfer](https://mlsanity.com/paper/2002.12489/) (2020)
- [Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences](https://mlsanity.com/paper/2108.07422/) (2021)
- [Learning Discriminative Representations for Skeleton Based Action Recognition](https://mlsanity.com/paper/2303.03729/) (2023)
- [Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification](https://mlsanity.com/paper/2104.02862/) (2021)
- [A Survey on 3D Skeleton-Based Action Recognition Using Learning Method](https://mlsanity.com/paper/2002.05907/) (2020)
- [Multi-scale 3D Convolution Network for Video Based Person Re-Identification](https://mlsanity.com/paper/1811.07468/) (2018)
- [Shape-Erased Feature Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2304.04205/) (2023)
- [Learning Modal-Invariant and Temporal-Memory for Video-based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2208.02450/) (2022)
- [Global-Local Temporal Representations For Video Person Re-Identification](https://mlsanity.com/paper/1908.10049/) (2019)
- [An Efficient Framework for Visible-Infrared Cross Modality Person Re-Identification](https://mlsanity.com/paper/1907.06498/) (2019)
- [On Exploring Pose Estimation as an Auxiliary Learning Task for Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2201.03859/) (2022)

## Abstract (arXiv)

Video-based visible-infrared person re-identification (VVI-ReID) is challenging due to significant modality feature discrepancies. Spatial-temporal information in videos is crucial, but the accuracy of spatial-temporal information is often influenced by issues like low quality and occlusions in videos. Existing methods mainly focus on reducing modality differences, but pay limited attention to improving spatial-temporal features, particularly for infrared videos. To address this, we propose a novel Skeleton-guided spatial-Temporal feAture leaRning (STAR) method for VVI-ReID. By using skeleton information, which is robust to issues such as poor image quality and occlusions, STAR improves the accuracy of spatial-temporal features in videos of both modalities. Specifically, STAR employs two levels of skeleton-guided strategies: frame level and sequence level. At the frame level, the robust structured skeleton information is used to refine the visual features of individual frames. At the sequence level, we design a feature aggregation mechanism based on skeleton key points graph, which learns the contribution of different body parts to spatial-temporal features, further enhancing the accuracy of global features. Experiments on benchmark datasets demonstrate that STAR outperforms state-of-the-art methods. Code will be open source soon.