# When Large Vision-Language Models Meet Person Re-Identification

arXiv: https://arxiv.org/abs/2411.18111 · HTML: https://mlsanity.com/paper/2411.18111/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 41 of its 46 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [A Survey on Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2402.00253/) (2024)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [Learning Discriminative Features with Multiple Granularities for Person Re-Identification](https://mlsanity.com/paper/1804.01438/) (2018)
- [Re-ranking Person Re-identification with k-reciprocal Encoding](https://mlsanity.com/paper/1701.08398/) (2017)
- [Joint Discriminative and Generative Learning for Person Re-identification](https://mlsanity.com/paper/1904.07223/) (2019)
- [CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels](https://mlsanity.com/paper/2211.13977/) (2022)
- [A Strong Baseline and Batch Normalization Neck for Deep Person Re-identification](https://mlsanity.com/paper/1906.08332/) (2019)
- [Relation-Aware Global Attention for Person Re-identification](https://mlsanity.com/paper/1904.02998/) (2019)

## Built on by (most cited first, 15 of 15)

- [IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identification](https://mlsanity.com/paper/2503.10324/) (2025)
- [DisenQ: Disentangling Q-Former for Activity-Biometrics](https://mlsanity.com/paper/2507.07262/) (2025)
- [Addressing the ID-Matching Challenge in Long Video Captioning](https://mlsanity.com/paper/2510.06973/) (2025)
- [Vision-Language Attribute Disentanglement and Reinforcement for Lifelong Person Re-Identification](https://mlsanity.com/paper/2603.19678/) (2026)
- [Towards Robust Text-to-Image Person Retrieval: Multi-View Reformulation for Semantic Compensation](https://mlsanity.com/paper/2604.18376/) (2026)
- [Following the Clues: Experiments on Person Re-ID using Cross-Modal Intelligence](https://mlsanity.com/paper/2507.01504/) (2025)
- [Seeing Across Time and Views: Multi-Temporal Cross-View Learning for Robust Video Person Re-Identification](https://mlsanity.com/paper/2511.02564/) (2025)
- [Zero-Shot Semantic Re-Identification for Autonomous Driving: A VLM Baseline Study](https://mlsanity.com/paper/2606.09362/) (2026)
- [Evolution of ReID: From Early Methods to LLM Integration](https://mlsanity.com/paper/2506.13039/) (2025)
- [Unleashing MLLMs on the Edge: A Unified Framework for Cross-Modal ReID via Adaptive SVD Distillation](https://mlsanity.com/paper/2602.12936/) (2026)
- [MSP-ReID: Hairstyle-Robust Cloth-Changing Person Re-Identification](https://mlsanity.com/paper/2603.01640/) (2026)
- [Beyond Visual Cues: Semantic-Driven Token Filtering and Expert Routing for Anytime Person ReID](https://mlsanity.com/paper/2604.15090/) (2026)
- [Multimodal LLM-Empowered Re-Ranking for Generalizable Person Re-Identification](https://mlsanity.com/paper/2606.16161/) (2026)
- [YesTrack: Referring Multi-Object Tracking via MLLM-based Yes/No Verification](https://mlsanity.com/paper/2609.02318/) (2026)
- [A Generative AI Integrated Multimodal Framework for Low-Latency Multi-Camera Person Re-Identification](https://mlsanity.com/paper/2609.14419/) (2026)

## Abstract (arXiv)

Large Vision-Language Models (LVLMs) that incorporate visual models and large language models have achieved impressive results across cross-modal understanding and reasoning tasks. In recent years, person re-identification (ReID) has also started to explore cross-modal semantics to improve the accuracy of identity recognition. However, effectively utilizing LVLMs for ReID remains an open challenge. While LVLMs operate under a generative paradigm by predicting the next output word, ReID requires the extraction of discriminative identity features to match pedestrians across cameras. In this paper, we propose LVLM-ReID, a novel framework that harnesses the strengths of LVLMs to promote ReID. Specifically, we employ instructions to guide the LVLM in generating one semantic token that encapsulates key appearance semantics from the person image. This token is further refined through our Semantic-Guided Interaction (SGI) module, establishing a reciprocal interaction between the semantic token and visual tokens. Ultimately, the reinforced semantic token serves as the representation of pedestrian identity. Our framework integrates the semantic understanding and generation capabilities of LVLM into end-to-end ReID training, allowing LVLM to capture rich semantic cues during both training and inference. LVLM-ReID achieves competitive results on multiple benchmarks without additional image-text annotations, demonstrating the potential of LVLM-generated semantics to advance person ReID.