# Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions

arXiv: https://arxiv.org/abs/2306.07520 · HTML: https://mlsanity.com/paper/2306.07520/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 81 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 30 of 55)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention](https://mlsanity.com/paper/2303.16199/) (2023)
- [VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding](https://mlsanity.com/paper/2109.14084/) (2021)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Language Is Not All You Need: Aligning Perception with Language Models](https://mlsanity.com/paper/2302.14045/) (2023)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [Person Transfer GAN to Bridge Domain Gap for Person Re-Identification](https://mlsanity.com/paper/1711.08565/) (2017)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training](https://mlsanity.com/paper/2005.00200/) (2020)
- [Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro](https://mlsanity.com/paper/1701.07717/) (2017)
- [Self-Supervised MultiModal Versatile Networks](https://mlsanity.com/paper/2006.16228/) (2020)
- [UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation](https://mlsanity.com/paper/2002.06353/) (2020)
- [Harmonious Attention Network for Person Re-Identification](https://mlsanity.com/paper/1802.08122/) (2018)
- [Person Re-identification: Past, Present and Future](https://mlsanity.com/paper/1610.02984/) (2016)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [Person Search with Natural Language Description](https://mlsanity.com/paper/1702.05729/) (2017)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [Relation-Aware Global Attention for Person Re-identification](https://mlsanity.com/paper/1904.02998/) (2019)
- [VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding](https://mlsanity.com/paper/2105.09996/) (2021)
- [COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning](https://mlsanity.com/paper/2011.00597/) (2020)
- [Clothes-Changing Person Re-identification with RGB Modality Only](https://mlsanity.com/paper/2204.06890/) (2022)

## Built on by (most cited first, 30 of 46)

- [IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identification](https://mlsanity.com/paper/2503.10324/) (2025)
- [Transformer for Object Re-Identification: A Survey](https://mlsanity.com/paper/2401.06960/) (2024)
- [ChatReID: Open-ended Interactive Person Retrieval via Hierarchical Progressive Tuning for Vision Language Models](https://mlsanity.com/paper/2502.19958/) (2025)
- [SeCap: Self-Calibrating and Adaptive Prompts for Cross-view Person Re-Identification in Aerial-Ground Networks](https://mlsanity.com/paper/2503.06965/) (2025)
- [AG-VPReID: A Challenging Large-Scale Benchmark for Aerial-Ground Video-based Person Re-Identification](https://mlsanity.com/paper/2503.08121/) (2025)
- [Human-Centric Foundation Models: Perception, Generation and Agentic Modeling](https://mlsanity.com/paper/2502.08556/) (2025)
- [DIFFER: Disentangling Identity Features via Semantic Cues for Clothes-Changing Person Re-ID](https://mlsanity.com/paper/2503.22912/) (2025)
- [ReID5o: Achieving Omni Multi-modal Person Re-identification in a Single Model](https://mlsanity.com/paper/2506.09385/) (2025)
- [CCUP: A Controllable Synthetic Data Generation Pipeline for Pretraining Cloth-Changing Person Re-Identification Models](https://mlsanity.com/paper/2410.13567/) (2024)
- [Towards All-in-One Medical Image Re-Identification](https://mlsanity.com/paper/2503.08173/) (2025)
- [Colors See Colors Ignore: Clothes Changing ReID with Color Disentanglement](https://mlsanity.com/paper/2507.07230/) (2025)
- [Exploring Part-Informed Visual-Language Learning for Person Re-Identification](https://mlsanity.com/paper/2308.02738/) (2023)
- [Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification](https://mlsanity.com/paper/2405.17790/) (2024)
- [Generalizable Object Re-Identification via Visual In-Context Prompting](https://mlsanity.com/paper/2508.21222/) (2025)
- [AnomalyLMM: Bridging Generative Knowledge and Discriminative Retrieval for Text-Based Person Anomaly Search](https://mlsanity.com/paper/2509.04376/) (2025)
- [One-Shot Knowledge Transfer for Scalable Person Re-Identification](https://mlsanity.com/paper/2511.06016/) (2025)
- [DisenQ: Disentangling Q-Former for Activity-Biometrics](https://mlsanity.com/paper/2507.07262/) (2025)
- [MOS: Mitigating Optical-SAR Modality Gap for Cross-Modal Ship Re-Identification](https://mlsanity.com/paper/2512.03404/) (2025)
- [BIT: Matching-based Bi-directional Interaction Transformation Network for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2603.14243/) (2026)
- [PSDiff: Diffusion Model for Person Search with Iterative and Collaborative Refinement](https://mlsanity.com/paper/2309.11125/) (2023)
- [VILLS -- Video-Image Learning to Learn Semantics for Person Re-Identification](https://mlsanity.com/paper/2311.17074/) (2023)
- [Multi-Domain Biometric Recognition using Body Embeddings](https://mlsanity.com/paper/2503.10931/) (2025)
- [One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models](https://mlsanity.com/paper/2507.07709/) (2025)
- [ReText: Text Boosts Generalization in Image-Based Person Re-identification](https://mlsanity.com/paper/2602.05785/) (2026)
- [AR2-4FV: Anchored Referring and Re-identification for Long-Term Grounding in Fixed-View Videos](https://mlsanity.com/paper/2603.07758/) (2026)
- [CMCC-ReID: Cross-Modality Clothing-Change Person Re-Identification](https://mlsanity.com/paper/2604.02808/) (2026)
- [Robust Onion: Peeling Open Vocab Object Detectors Under Noise](https://mlsanity.com/paper/2606.26734/) (2026)
- [See What You Seek: Semantic Contextual Integration for Cloth-Changing Person Re-Identification](https://mlsanity.com/paper/2412.01345/) (2024)
- [Adept: Annotation-Denoising Auxiliary Tasks with Discrete Cosine Transform Map and Keypoint for Human-Centric Pretraining](https://mlsanity.com/paper/2504.20800/) (2025)
- [Cross-Spectral Body Recognition with Side Information Embedding: Benchmarks on LLCM and Analyzing Range-Induced Occlusions on IJB-MDF](https://mlsanity.com/paper/2506.08953/) (2025)

## Abstract (arXiv)

Human intelligence can retrieve any person according to both visual and language descriptions. However, the current computer vision community studies specific person re-identification (ReID) tasks in different scenarios separately, which limits the applications in the real world. This paper strives to resolve this problem by proposing a new instruct-ReID task that requires the model to retrieve images according to the given image or language instructions. Our instruct-ReID is a more general ReID setting, where existing 6 ReID tasks can be viewed as special cases by designing different instructions. We propose a large-scale OmniReID benchmark and an adaptive triplet loss as a baseline method to facilitate research in this new setting. Experimental results show that the proposed multi-purpose ReID model, trained on our OmniReID benchmark without fine-tuning, can improve +0.5%, +0.6%, +7.7% mAP on Market1501, MSMT17, CUHK03 for traditional ReID, +6.4%, +7.1%, +11.2% mAP on PRCC, VC-Clothes, LTCC for clothes-changing ReID, +11.7% mAP on COCAS+ real2 for clothes template based clothes-changing ReID when using only RGB images, +24.9% mAP on COCAS+ real2 for our newly defined language-instructed ReID, +4.3% on LLCM for visible-infrared ReID, +2.6% on CUHK-PEDES for text-to-image ReID. The datasets, the model, and code will be available at https://github.com/hwz-zju/Instruct-ReID.