# CMTR: Cross-modality Transformer for Visible-infrared Person Re-identification

arXiv: https://arxiv.org/abs/2110.08994 · HTML: https://mlsanity.com/paper/2110.08994/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 69 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [Learning Discriminative Features with Multiple Granularities for Person Re-Identification](https://mlsanity.com/paper/1804.01438/) (2018)
- [Person Re-identification: Past, Present and Future](https://mlsanity.com/paper/1610.02984/) (2016)
- [A Strong Baseline and Batch Normalization Neck for Deep Person Re-identification](https://mlsanity.com/paper/1906.08332/) (2019)
- [Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2007.09314/) (2020)
- [RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment](https://mlsanity.com/paper/1910.05839/) (2019)
- [Pose Recognition with Cascade Transformers](https://mlsanity.com/paper/2104.06976/) (2021)
- [Cross-modality Person re-identification with Shared-Specific Feature Transfer](https://mlsanity.com/paper/2002.12489/) (2020)
- [FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification](https://mlsanity.com/paper/1810.02936/) (2018)
- [Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification](https://mlsanity.com/paper/2002.04114/) (2020)
- [Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/1912.01230/) (2019)
- [Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences](https://mlsanity.com/paper/2108.07422/) (2021)
- [Neural Feature Search for RGB-Infrared Person Re-Identification](https://mlsanity.com/paper/2104.02366/) (2021)
- [Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification](https://mlsanity.com/paper/2104.02862/) (2021)
- [CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2101.08467/) (2021)
- [Beyond Triplet Loss: Person Re-identification with Fine-grained Difference-aware Pairwise Loss](https://mlsanity.com/paper/2009.10295/) (2020)

## Built on by (most cited first, 16 of 16)

- [Learning Progressive Modality-shared Transformers for Effective Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2212.00226/) (2022)
- [Transformer for Object Re-Identification: A Survey](https://mlsanity.com/paper/2401.06960/) (2024)
- [Cross-Modality High-Frequency Transformer for MR Image Super-Resolution](https://mlsanity.com/paper/2203.15314/) (2022)
- [CLIP-Driven Semantic Discovery Network for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2401.05806/) (2024)
- [ICPL-ReID: Identity-Conditional Prompt Learning for Multi-Spectral Object Re-Identification](https://mlsanity.com/paper/2505.17821/) (2025)
- [Cross-modal Ship Re-Identification via Optical and SAR Imagery: A Novel Dataset and Method](https://mlsanity.com/paper/2506.22027/) (2025)
- [Modality-Transition Representation Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2511.02685/) (2025)
- [Robust Duality Learning for Unsupervised Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2505.02549/) (2025)
- [BIT: Matching-based Bi-directional Interaction Transformation Network for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2603.14243/) (2026)
- [CLIP4VI-ReID: Learning Modality-shared Representations via CLIP Semantic Bridge for Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2511.10309/) (2025)
- [CMCC-ReID: Cross-Modality Clothing-Change Person Re-Identification](https://mlsanity.com/paper/2604.02808/) (2026)
- [Transferring Modality-Aware Pedestrian Attentive Learning for Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2312.07021/) (2023)
- [Cross-Spectral Body Recognition with Side Information Embedding: Benchmarks on LLCM and Analyzing Range-Induced Occlusions on IJB-MDF](https://mlsanity.com/paper/2506.08953/) (2025)
- [Temperature calibration of surface emissivities with an improved thermal image enhancement network](https://mlsanity.com/paper/2506.16803/) (2025)
- [Identity Clue Refinement and Enhancement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2512.04522/) (2025)
- [Blurring Modal Boundaries: A Unified Survey from Single- to Multi-Modal Person Re-ldentification](https://mlsanity.com/paper/2607.14821/) (2026)

## Abstract (arXiv)

Visible-infrared cross-modality person re-identification is a challenging ReID task, which aims to retrieve and match the same identity's images between the heterogeneous visible and infrared modalities. Thus, the core of this task is to bridge the huge gap between these two modalities. The existing convolutional neural network-based methods mainly face the problem of insufficient perception of modalities' information, and can not learn good discriminative modality-invariant embeddings for identities, which limits their performance. To solve these problems, we propose a cross-modality transformer-based method (CMTR) for the visible-infrared person re-identification task, which can explicitly mine the information of each modality and generate better discriminative features based on it. Specifically, to capture modalities' characteristics, we design the novel modality embeddings, which are fused with token embeddings to encode modalities' information. Furthermore, to enhance representation of modality embeddings and adjust matching embeddings' distribution, we propose a modality-aware enhancement loss based on the learned modalities' information, reducing intra-class distance and enlarging inter-class distance. To our knowledge, this is the first work of applying transformer network to the cross-modality re-identification task. We implement extensive experiments on the public SYSU-MM01 and RegDB datasets, and our proposed CMTR model's performance significantly surpasses existing outstanding CNN-based methods.