# MLLMReID: Multimodal Large Language Model-based Person Re-identification

arXiv: https://arxiv.org/abs/2401.13201 · HTML: https://mlsanity.com/paper/2401.13201/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 32 of its 41 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 30 of 32)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Person Transfer GAN to Bridge Domain Gap for Person Re-Identification](https://mlsanity.com/paper/1711.08565/) (2017)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019)
- [Learning Discriminative Features with Multiple Granularities for Person Re-Identification](https://mlsanity.com/paper/1804.01438/) (2018)
- [Multimodal Foundation Models: From Specialists to General-Purpose Assistants](https://mlsanity.com/paper/2309.10020/) (2023)
- [CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels](https://mlsanity.com/paper/2211.13977/) (2022)
- [Person Search with Natural Language Description](https://mlsanity.com/paper/1702.05729/) (2017)
- [ABD-Net: Attentive but Diverse Person Re-Identification](https://mlsanity.com/paper/1908.01114/) (2019)
- [Identity-Guided Human Semantic Parsing for Person Re-Identification](https://mlsanity.com/paper/2007.13467/) (2020)
- [Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification](https://mlsanity.com/paper/2012.00417/) (2020)
- [Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification](https://mlsanity.com/paper/2107.12666/) (2021)
- [High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification](https://mlsanity.com/paper/2003.08177/) (2020)
- [DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrieval](https://mlsanity.com/paper/2109.05534/) (2021)
- [Occluded Person Re-identification](https://mlsanity.com/paper/1804.02792/) (2018)
- [See Finer, See More: Implicit Modality Alignment for Text-based Person Retrieval](https://mlsanity.com/paper/2208.08608/) (2022)
- [UniHCP: A Unified Model for Human-Centric Perceptions](https://mlsanity.com/paper/2303.02936/) (2023)
- [Graph Sampling Based Deep Metric Learning for Generalizable Person Re-Identification](https://mlsanity.com/paper/2104.01546/) (2021)
- [Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association](https://mlsanity.com/paper/1808.01571/) (2018)
- [Region Generation and Assessment Network for Occluded Person Re-Identification](https://mlsanity.com/paper/2309.03558/) (2023)

## Built on by (most cited first, 8 of 8)

- [ChatReID: Open-ended Interactive Person Retrieval via Hierarchical Progressive Tuning for Vision Language Models](https://mlsanity.com/paper/2502.19958/) (2025)
- [DIFFER: Disentangling Identity Features via Semantic Cues for Clothes-Changing Person Re-ID](https://mlsanity.com/paper/2503.22912/) (2025)
- [Vision-Language Attribute Disentanglement and Reinforcement for Lifelong Person Re-Identification](https://mlsanity.com/paper/2603.19678/) (2026)
- [Towards Robust Text-to-Image Person Retrieval: Multi-View Reformulation for Semantic Compensation](https://mlsanity.com/paper/2604.18376/) (2026)
- [Evolution of ReID: From Early Methods to LLM Integration](https://mlsanity.com/paper/2506.13039/) (2025)
- [Harnessing Object Grounding for Time-Sensitive Video Understanding](https://mlsanity.com/paper/2509.06335/) (2025)
- [Unleashing MLLMs on the Edge: A Unified Framework for Cross-Modal ReID via Adaptive SVD Distillation](https://mlsanity.com/paper/2602.12936/) (2026)
- [AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes](https://mlsanity.com/paper/2606.02724/) (2026)

## Abstract (arXiv)

Multimodal large language models (MLLM) have achieved satisfactory results in many tasks. However, their performance in the task of ReID (ReID) has not been explored to date. This paper will investigate how to adapt them for the task of ReID. An intuitive idea is to fine-tune MLLM with ReID image-text datasets, and then use their visual encoder as a backbone for ReID. However, there still exist two apparent issues: (1) Designing instructions for ReID, MLLMs may overfit specific instructions, and designing a variety of instructions will lead to higher costs. (2) When fine-tuning the visual encoder of a MLLM, it is not trained synchronously with the ReID task. As a result, the effectiveness of the visual encoder fine-tuning cannot be directly reflected in the performance of the ReID task. To address these problems, this paper proposes MLLMReID: Multimodal Large Language Model-based ReID. Firstly, we proposed Common Instruction, a simple approach that leverages the essence ability of LLMs to continue writing, avoiding complex and diverse instruction design. Secondly, we propose a multi-task learning-based synchronization module to ensure that the visual encoder of the MLLM is trained synchronously with the ReID task. The experimental results demonstrate the superiority of our method.