# Learning Modal-Invariant and Temporal-Memory for Video-based Visible-Infrared Person Re-Identification

arXiv: https://arxiv.org/abs/2208.02450 · HTML: https://mlsanity.com/paper/2208.02450/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Harmonious Attention Network for Person Re-Identification](https://mlsanity.com/paper/1802.08122/) (2018)
- [Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2007.09314/) (2020)
- [RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment](https://mlsanity.com/paper/1910.05839/) (2019)
- [Cross-modality Person re-identification with Shared-Specific Feature Transfer](https://mlsanity.com/paper/2002.12489/) (2020)
- [Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/1912.01230/) (2019)
- [Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences](https://mlsanity.com/paper/2108.07422/) (2021)
- [Appearance-Preserving 3D Convolution for Video-based Person Re-identification](https://mlsanity.com/paper/2007.08434/) (2020)
- [Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification](https://mlsanity.com/paper/2104.02862/) (2021)
- [Unsupervised Person Re-identification by Deep Learning Tracklet Association](https://mlsanity.com/paper/1809.02874/) (2018)
- [Multi-scale 3D Convolution Network for Video Based Person Re-Identification](https://mlsanity.com/paper/1811.07468/) (2018)
- [Jointly Attentive Spatial-Temporal Pooling Networks for Video-based Person Re-Identification](https://mlsanity.com/paper/1708.02286/) (2017)
- [Video-based Person Re-identification with Spatial and Temporal Memory Networks](https://mlsanity.com/paper/2108.09039/) (2021)
- [A Novel Unsupervised Camera-aware Domain Adaptation Framework for Person Re-identification](https://mlsanity.com/paper/1904.03425/) (2019)
- [Spatial and Temporal Mutual Promotion for Video-based Person Re-identification](https://mlsanity.com/paper/1812.10305/) (2018)
- [Spatio-Temporal Representation Factorization for Video-based Person Re-Identification](https://mlsanity.com/paper/2107.11878/) (2021)
- [Revisiting Temporal Modeling for Video-based Person ReID](https://mlsanity.com/paper/1805.02104/) (2018)
- [Dual Gaussian-based Variational Subspace Disentanglement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2008.02520/) (2020)
- [Recover and Identify: A Generative Dual Model for Cross-Resolution Person Re-Identification](https://mlsanity.com/paper/1908.06052/) (2019)
- [Dual-Stream Reciprocal Disentanglement Learning for Domain Adaptation Person Re-Identification](https://mlsanity.com/paper/2106.13929/) (2021)

## Built on by (most cited first, 25 of 25)

- [Shape-Erased Feature Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2304.04205/) (2023)
- [M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition](https://mlsanity.com/paper/2308.03063/) (2023)
- [CLIP-Driven Semantic Discovery Network for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2401.05806/) (2024)
- [A Versatile Framework for Multi-scene Person Re-identification](https://mlsanity.com/paper/2403.11121/) (2024)
- [Exploring Homogeneous and Heterogeneous Consistent Label Associations for Unsupervised Visible-Infrared Person ReID](https://mlsanity.com/paper/2402.00672/) (2024)
- [Exploring Fine-Grained Representation and Recomposition for Cloth-Changing Person Re-Identification](https://mlsanity.com/paper/2308.10692/) (2023)
- [Video-based Visible-Infrared Person Re-Identification with Auxiliary Samples](https://mlsanity.com/paper/2311.15571/) (2023)
- [Camera-LiDAR Cross-modality Gait Recognition](https://mlsanity.com/paper/2407.02038/) (2024)
- [Cross-Modality Gait Recognition: Bridging LiDAR and Camera Modalities for Human Identification](https://mlsanity.com/paper/2404.04120/) (2024)
- [Shape-centered Representation Learning for Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2310.17952/) (2023)
- [Video-Level Language-Driven Video-Based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2506.02439/) (2025)
- [Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification](https://mlsanity.com/paper/2501.16811/) (2025)
- [Multi-modal Multi-platform Person Re-Identification: Benchmark and Method](https://mlsanity.com/paper/2503.17096/) (2025)
- [Diverse Representation Embedding for Lifelong Person Re-Identification](https://mlsanity.com/paper/2403.16003/) (2024)
- [Adversarial Self-Attack Defense and Spatial-Temporal Relation Mining for Visible-Infrared Video Person Re-Identification](https://mlsanity.com/paper/2307.03903/) (2023)
- [YYDS: Visible-Infrared Person Re-Identification with Coarse Descriptions](https://mlsanity.com/paper/2403.04183/) (2024)
- [Person detection and re-identification in open-world settings of retail stores and public spaces](https://mlsanity.com/paper/2505.00772/) (2025)
- [Causal Bootstrapped Alignment for Unsupervised Video-Based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2604.15631/) (2026)
- [Prototype-Guided Text-based Person Search based on Rich Chinese Descriptions](https://mlsanity.com/paper/2312.14834/) (2023)
- [Skeleton-Guided Spatial-Temporal Feature Learning for Video-Based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2411.11069/) (2024)
- [FDCT: Frequency-Aware Decomposition and Cross-Modal Token-Alignment for Multi-Sensor Target Classification](https://mlsanity.com/paper/2503.09873/) (2025)
- [AG-VPReID.VIR: Bridging Aerial and Ground Platforms for Video-based Visible-Infrared Person Re-ID](https://mlsanity.com/paper/2507.17995/) (2025)
- [Attentive Convolution: Unifying the Expressivity of Self-Attention with Convolutional Efficiency](https://mlsanity.com/paper/2510.20092/) (2025)
- [Learning Language-Driven Sequence-Level Modal-Invariant Representations for Video-Based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2601.12062/) (2026)
- [Temporal Prototyping and Hierarchical Alignment for Unsupervised Video-based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2604.21324/) (2026)

## Abstract (arXiv)

Thanks for the cross-modal retrieval techniques, visible-infrared (RGB-IR) person re-identification (Re-ID) is achieved by projecting them into a common space, allowing person Re-ID in 24-hour surveillance systems. However, with respect to the probe-to-gallery, almost all existing RGB-IR based cross-modal person Re-ID methods focus on image-to-image matching, while the video-to-video matching which contains much richer spatial- and temporal-information remains under-explored. In this paper, we primarily study the video-based cross-modal person Re-ID method. To achieve this task, a video-based RGB-IR dataset is constructed, in which 927 valid identities with 463,259 frames and 21,863 tracklets captured by 12 RGB/IR cameras are collected. Based on our constructed dataset, we prove that with the increase of frames in a tracklet, the performance does meet more enhancement, demonstrating the significance of video-to-video matching in RGB-IR person Re-ID. Additionally, a novel method is further proposed, which not only projects two modalities to a modal-invariant subspace, but also extracts the temporal-memory for motion-invariant. Thanks to these two strategies, much better results are achieved on our video-based cross-modal person Re-ID. The code and dataset are released at: https://github.com/VCMproject233/MITML.