# Exploring Modality-shared Appearance Features and Modality-invariant Relation Features for Cross-modality Person Re-Identification

arXiv: https://arxiv.org/abs/2104.11539 · HTML: https://mlsanity.com/paper/2104.11539/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 79 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Beyond triplet loss: a deep quadruplet network for person re-identification](https://mlsanity.com/paper/1704.01719/) (2017)
- [Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2007.09314/) (2020)
- [RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment](https://mlsanity.com/paper/1910.05839/) (2019)
- [Cross-modality Person re-identification with Shared-Specific Feature Transfer](https://mlsanity.com/paper/2002.12489/) (2020)
- [Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/1912.01230/) (2019)
- [Pose Invariant Embedding for Deep Person Re-identification](https://mlsanity.com/paper/1701.07732/) (2017)
- [Deep Representation Learning with Part Loss for Person Re-Identification](https://mlsanity.com/paper/1707.00798/) (2017)
- [Hetero-Center Loss for Cross-Modality Person Re-Identification](https://mlsanity.com/paper/1910.09830/) (2019)
- [Pedestrian Alignment Network for Large-scale Person Re-identification](https://mlsanity.com/paper/1707.00408/) (2017)
- [Dual Gaussian-based Variational Subspace Disentanglement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2008.02520/) (2020)
- [SFANet: A Spectrum-aware Feature Augmentation Network for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2102.12137/) (2021)
- [Enhancing the Discriminative Feature Learning for Visible-Thermal Cross-Modality Person Re-Identification](https://mlsanity.com/paper/1907.09659/) (2019)
- [HPILN: A feature learning framework for cross-modality person re-identification](https://mlsanity.com/paper/1906.03142/) (2019)
- [Bi-directional Exponential Angular Triplet Loss for RGB-Infrared Person Re-Identification](https://mlsanity.com/paper/2006.00878/) (2020)
- [RGB-IR Cross-modality Person ReID based on Teacher-Student GAN Model](https://mlsanity.com/paper/2007.07452/) (2020)
- [Attend to the Difference: Cross-Modality Person Re-identification via Contrastive Correlation](https://mlsanity.com/paper/1910.11656/) (2019)
- [An Efficient Framework for Visible-Infrared Cross Modality Person Re-Identification](https://mlsanity.com/paper/1907.06498/) (2019)
- [Attributes-aided Part Detection and Refinement for Person Re-identification](https://mlsanity.com/paper/1902.10528/) (2019)
- [PaMM: Pose-aware Multi-shot Matching for Improving Person Re-identification](https://mlsanity.com/paper/1705.06011/) (2017)
- [Learning View-Specific Deep Networks for Person Re-Identification](https://mlsanity.com/paper/1803.11333/) (2018)
- [Review of Person Re-identification Techniques](https://mlsanity.com/paper/1511.02319/) (2015)

## Built on by (most cited first, 6 of 6)

- [A Versatile Framework for Multi-scene Person Re-identification](https://mlsanity.com/paper/2403.11121/) (2024)
- [Visible-Infrared Person Re-Identification via Patch-Mixed Cross-Modality Learning](https://mlsanity.com/paper/2302.08212/) (2023)
- [MIR-GAN: Refining Frame-Level Modality-Invariant Representations with Adversarial Network for Audio-Visual Speech Recognition](https://mlsanity.com/paper/2306.10567/) (2023)
- [Homogeneous and Heterogeneous Relational Graph for Visible-infrared Person Re-identification](https://mlsanity.com/paper/2109.08811/) (2021)
- [VILLS -- Video-Image Learning to Learn Semantics for Person Re-Identification](https://mlsanity.com/paper/2311.17074/) (2023)
- [Robust Ensemble Person Re-Identification via Orthogonal Fusion with Occlusion Handling](https://mlsanity.com/paper/2404.00107/) (2024)

## Abstract (arXiv)

Most existing cross-modality person re-identification works rely on discriminative modality-shared features for reducing cross-modality variations and intra-modality variations. Despite some initial success, such modality-shared appearance features cannot capture enough modality-invariant discriminative information due to a massive discrepancy between RGB and infrared images. To address this issue, on the top of appearance features, we further capture the modality-invariant relations among different person parts (referred to as modality-invariant relation features), which are the complement to those modality-shared appearance features and help to identify persons with similar appearances but different body shapes. To this end, a Multi-level Two-streamed Modality-shared Feature Extraction (MTMFE) sub-network is designed, where the modality-shared appearance features and modality-invariant relation features are first extracted in a shared 2D feature space and a shared 3D feature space, respectively. The two features are then fused into the final modality-shared features such that both cross-modality variations and intra-modality variations can be reduced. Besides, a novel cross-modality quadruplet loss is proposed to further reduce the cross-modality variations. Experimental results on several benchmark datasets demonstrate that our proposed method exceeds state-of-the-art algorithms by a noticeable margin.