# Cross-Spectral Attention for Unsupervised RGB-IR Face Verification and Person Re-identification

arXiv: https://arxiv.org/abs/2411.19215 · HTML: https://mlsanity.com/paper/2411.19215/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 71 references that are in the map.
- First posted on arXiv 28 Nov 2024.

## Builds on (most cited first, 30 of 32)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019)
- [Re-ranking Person Re-identification with k-reciprocal Encoding](https://mlsanity.com/paper/1701.08398/) (2017)
- [Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-identification](https://mlsanity.com/paper/1904.01990/) (2019)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2007.09314/) (2020)
- [Cross-modality Person re-identification with Shared-Specific Feature Transfer](https://mlsanity.com/paper/2002.12489/) (2020)
- [Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification](https://mlsanity.com/paper/2002.04114/) (2020)
- [Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/1912.01230/) (2019)
- [Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences](https://mlsanity.com/paper/2108.07422/) (2021)
- [Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification](https://mlsanity.com/paper/2104.02862/) (2021)
- [Unsupervised Sketch-to-Photo Synthesis](https://mlsanity.com/paper/1909.08313/) (2019)
- [Unsupervised Domain Adaptation in the Dissimilarity Space for Person Re-identification](https://mlsanity.com/paper/2007.13890/) (2020)
- [Pose-Robust Face Recognition via Deep Residual Equivariant Mapping](https://mlsanity.com/paper/1803.00839/) (2018)
- [Implicit Discriminative Knowledge Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2403.11708/) (2024)
- [A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset](https://mlsanity.com/paper/2101.02637/) (2021)
- [Hybrid Dynamic Contrast and Probability Distillation for Unsupervised Person Re-Id](https://mlsanity.com/paper/2109.14157/) (2021)
- [Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis](https://mlsanity.com/paper/1904.07344/) (2019)
- [Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistles](https://mlsanity.com/paper/2210.06529/) (2022)
- [An Efficient Hardware Accelerator for Structured Sparse Convolutional Neural Networks on FPGAs](https://mlsanity.com/paper/2001.01955/) (2020)
- [Cross-Domain Identification for Thermal-to-Visible Face Recognition](https://mlsanity.com/paper/2008.08473/) (2020)
- [Generative Adversarial Network-based Synthesis of Visible Faces from Polarimetric Thermal Faces](https://mlsanity.com/paper/1708.02681/) (2017)
- [Unsupervised Learning of Face Representations](https://mlsanity.com/paper/1803.01260/) (2018)
- [HashReID: Dynamic Network with Binary Codes for Efficient Person Re-identification](https://mlsanity.com/paper/2308.11900/) (2023)

## Abstract (arXiv)

Cross-spectral biometrics, such as matching imagery of faces or persons from visible (RGB) and infrared (IR) bands, have rapidly advanced over the last decade due to increasing sensitivity, size, quality, and ubiquity of IR focal plane arrays and enhanced analytics beyond the visible spectrum. Current techniques for mitigating large spectral disparities between RGB and IR imagery often include learning a discriminative common subspace by exploiting precisely curated data acquired from multiple spectra. Although there are challenges with determining robust architectures for extracting common information, a critical limitation for supervised methods is poor scalability in terms of acquiring labeled data. Therefore, we propose a novel unsupervised cross-spectral framework that combines (1) a new pseudo triplet loss with cross-spectral voting, (2) a new cross-spectral attention network leveraging multiple subspaces, and (3) structured sparsity to perform more discriminative cross-spectral clustering. We extensively compare our proposed RGB-IR biometric learning framework (and its individual components) with recent and previous state-of-the-art models on two challenging benchmark datasets: DEVCOM Army Research Laboratory Visible-Thermal Face Dataset (ARL-VTF) and RegDB person re-identification dataset, and, in some cases, achieve performance superior to completely supervised methods.