# CLIP-Driven Semantic Discovery Network for Visible-Infrared Person Re-Identification

arXiv: https://arxiv.org/abs/2401.05806 · HTML: https://mlsanity.com/paper/2401.05806/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 83 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://mlsanity.com/paper/1903.07071/) (2019)
- [CRIS: CLIP-Driven Referring Image Segmentation](https://mlsanity.com/paper/2111.15174/) (2021)
- [CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels](https://mlsanity.com/paper/2211.13977/) (2022)
- [ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2212.03588/) (2022)
- [Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2007.09314/) (2020)
- [RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment](https://mlsanity.com/paper/1910.05839/) (2019)
- [Exploiting Unlabeled Data with Vision and Language Models for Object Detection](https://mlsanity.com/paper/2207.08954/) (2022)
- [CLIP-Driven Fine-grained Text-Image Person Re-identification](https://mlsanity.com/paper/2210.10276/) (2022)
- [CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets](https://mlsanity.com/paper/2302.02551/) (2023)
- [Diverse Embedding Expansion Network and Low-Light Cross-Modality Benchmark for Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2303.14481/) (2023)
- [FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification](https://mlsanity.com/paper/1810.02936/) (2018)
- [Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification](https://mlsanity.com/paper/2002.04114/) (2020)
- [Pose-Normalized Image Generation for Person Re-identification](https://mlsanity.com/paper/1712.02225/) (2017)
- [Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/1912.01230/) (2019)
- [Learning by Aligning: Visible-Infrared Person Re-identification using Cross-Modal Correspondences](https://mlsanity.com/paper/2108.07422/) (2021)
- [AANet: Attribute Attention Network for Person Re-Identifications](https://mlsanity.com/paper/1912.09021/) (2019)
- [Neural Feature Search for RGB-Infrared Person Re-Identification](https://mlsanity.com/paper/2104.02366/) (2021)

## Built on by (most cited first, 19 of 19)

- [Rethinking Early-Fusion Strategies for Improved Multispectral Object Detection](https://mlsanity.com/paper/2405.16038/) (2024)
- [Video-Level Language-Driven Video-Based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2506.02439/) (2025)
- [Multi-modal Multi-platform Person Re-Identification: Benchmark and Method](https://mlsanity.com/paper/2503.17096/) (2025)
- [Diverse Representation Embedding for Lifelong Person Re-Identification](https://mlsanity.com/paper/2403.16003/) (2024)
- [Diverse Semantics-Guided Feature Alignment and Decoupling for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2505.00619/) (2025)
- [Weakly Supervised Visible-Infrared Person Re-Identification via Heterogeneous Expert Collaborative Consistency Learning](https://mlsanity.com/paper/2507.12942/) (2025)
- [From Limited Labels to Open Domains:An Efficient Learning Method for Drone-view Geo-Localization](https://mlsanity.com/paper/2503.07520/) (2025)
- [CLIP4VI-ReID: Learning Modality-shared Representations via CLIP Semantic Bridge for Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2511.10309/) (2025)
- [Causal Bootstrapped Alignment for Unsupervised Video-Based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2604.15631/) (2026)
- [See What You Seek: Semantic Contextual Integration for Cloth-Changing Person Re-Identification](https://mlsanity.com/paper/2412.01345/) (2024)
- [Embedding and Enriching Explicit Semantics for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2412.08406/) (2024)
- [FitPro: A Zero-Shot Framework for Interactive Text-based Pedestrian Retrieval in Open World](https://mlsanity.com/paper/2509.16674/) (2025)
- [Learning Language-Driven Sequence-Level Modal-Invariant Representations for Video-Based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2601.12062/) (2026)
- [DiffCrossGait: Trajectory-Level Alignment for 2D-3D Cross-Modal Gait Recognition via Latent Diffusion](https://mlsanity.com/paper/2606.00153/) (2026)
- [Bridging Day and Night: Unsupervised Cross-Domain Re-Identification with Synergistic Prompt and Prototype Learning](https://mlsanity.com/paper/2606.12258/) (2026)
- [GeoFlow: Geo-Aware Modeling of Inter-Area Relationships in Origin-Destination Flow Prediction and Generation](https://mlsanity.com/paper/2607.05257/) (2026)
- [Structural-Semantic Reciprocal Learning for Unsupervised Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2607.15220/) (2026)
- [Dual-Space Modality Consistency Learning for Universal Cross-Modal Re-Identification](https://mlsanity.com/paper/2608.06943/) (2026)
- [Multi-scale Decomposed Convolution Refinement Network for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2608.16015/) (2026)

## Abstract (arXiv)

Visible-infrared person re-identification (VIReID) primarily deals with matching identities across person images from different modalities. Due to the modality gap between visible and infrared images, cross-modality identity matching poses significant challenges. Recognizing that high-level semantics of pedestrian appearance, such as gender, shape, and clothing style, remain consistent across modalities, this paper intends to bridge the modality gap by infusing visual features with high-level semantics. Given the capability of CLIP to sense high-level semantic information corresponding to visual representations, we explore the application of CLIP within the domain of VIReID. Consequently, we propose a CLIP-Driven Semantic Discovery Network (CSDN) that consists of Modality-specific Prompt Learner, Semantic Information Integration (SII), and High-level Semantic Embedding (HSE). Specifically, considering the diversity stemming from modality discrepancies in language descriptions, we devise bimodal learnable text tokens to capture modality-private semantic information for visible and infrared images, respectively. Additionally, acknowledging the complementary nature of semantic details across different modalities, we integrate text features from the bimodal language descriptions to achieve comprehensive semantics. Finally, we establish a connection between the integrated text features and the visual features across modalities. This process embed rich high-level semantic information into visual representations, thereby promoting the modality invariance of visual representations. The effectiveness and superiority of our proposed CSDN over existing methods have been substantiated through experimental evaluations on multiple widely used benchmarks. The code will be released at urlhttps://github.com/nengdong96/CSDN.