# Temporal Prototyping and Hierarchical Alignment for Unsupervised Video-based Visible-Infrared Person Re-Identification

arXiv: https://arxiv.org/abs/2604.21324 · HTML: https://mlsanity.com/paper/2604.21324/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 66 references that are in the map.
- First posted on arXiv 23 Apr 2026.

## Builds on (most cited first, 23 of 23)

- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2007.09314/) (2020)
- [Cluster Contrast for Unsupervised Person Re-Identification](https://mlsanity.com/paper/2103.11568/) (2021)
- [Cross-modality Person re-identification with Shared-Specific Feature Transfer](https://mlsanity.com/paper/2002.12489/) (2020)
- [Diverse Embedding Expansion Network and Low-Light Cross-Modality Benchmark for Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2303.14481/) (2023)
- [Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences](https://mlsanity.com/paper/2108.07422/) (2021)
- [Appearance-Preserving 3D Convolution for Video-based Person Re-identification](https://mlsanity.com/paper/2007.08434/) (2020)
- [Multi-Level Factorisation Net for Person Re-Identification](https://mlsanity.com/paper/1803.09132/) (2018)
- [Unsupervised Tracklet Person Re-Identification](https://mlsanity.com/paper/1903.00535/) (2019)
- [Temporal Complementary Learning for Video Person Re-Identification](https://mlsanity.com/paper/2007.09357/) (2020)
- [STA: Spatial-Temporal Attention for Large-Scale Video-based Person Re-Identification](https://mlsanity.com/paper/1811.04129/) (2018)
- [Unsupervised Person Re-identification by Deep Learning Tracklet Association](https://mlsanity.com/paper/1809.02874/) (2018)
- [Multi-scale 3D Convolution Network for Video Based Person Re-Identification](https://mlsanity.com/paper/1811.07468/) (2018)
- [Learning Progressive Modality-shared Transformers for Effective Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2212.00226/) (2022)
- [Spatial and Temporal Mutual Promotion for Video-based Person Re-identification](https://mlsanity.com/paper/1812.10305/) (2018)
- [Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2401.06825/) (2024)
- [Learning Modal-Invariant and Temporal-Memory for Video-based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2208.02450/) (2022)
- [Video-based Person Re-identification with Accumulative Motion Context](https://mlsanity.com/paper/1701.00193/) (2017)
- [Video-based Visible-Infrared Person Re-Identification with Auxiliary Samples](https://mlsanity.com/paper/2311.15571/) (2023)
- [Shape-centered Representation Learning for Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2310.17952/) (2023)
- [Unsupervised Video Person Re-identification via Noise and Hard frame Aware Clustering](https://mlsanity.com/paper/2106.05441/) (2021)
- [Towards Homogeneous Modality Learning and Multi-Granularity Information Exploration for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2204.04842/) (2022)
- [Adversarial Self-Attack Defense and Spatial-Temporal Relation Mining for Visible-Infrared Video Person Re-Identification](https://mlsanity.com/paper/2307.03903/) (2023)

## Abstract (arXiv)

Visible-infrared person re-identification (VI-ReID) enables cross-modality identity matching for all-day surveillance, yet existing methods predominantly focus on the image level or rely heavily on costly identity annotations. While video-based VI-ReID has recently emerged to exploit temporal dynamics for improved robustness, existing studies remain limited to supervised settings. Crucially, the unsupervised video VI-ReID problem, where models must learn from RGB and infrared tracklets without identity labels, remains largely unexplored despite its practical importance in real-world deployment. To bridge this gap, we propose HiTPro (Hierarchical Temporal Prototyping), a prototype-driven framework without explicit hard pseudo-label assignment for unsupervised video-based VI-ReID. HiTPro begins with an efficient Temporal-aware Feature Encoder that first extracts discriminative frame-level features and then aggregates them into a robust tracklet-level representation. Building upon these features, HiTPro first constructs reliable intra-camera prototypes via Intra-Camera Tracklet Prototyping by aggregating features from temporally partitioned sub-tracklets. Through Hierarchical Cross-Prototype Alignment, we perform a two-stage positive mining process: progressing from within-modality associations to cross-modality matching, enhanced by Dynamic Threshold Strategy and Soft Weight Assignment. Finally, {Hierarchical Contrastive Learning} progressively optimizes feature-prototype alignment across three levels: intra-camera discrimination, cross-camera same-modality consistency, and cross-modality invariance. Extensive experiments on HITSZ-VCM and BUPTCampus demonstrate that HiTPro achieves state-of-the-art performance under fully unsupervised settings, significantly outperforming adapted baselines and establishes a strong baseline for future research.