# CLIP-SCGI: Synthesized Caption-Guided Inversion for Person Re-Identification arXiv: https://arxiv.org/abs/2410.09382 ยท HTML: https://mlsanity.com/paper/2410.09382/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 39 of its 52 references that are in the map. - First posted on arXiv 12 Oct 2024. ## Placed in - Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/) ## Builds on (most cited first, 30 of 39) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023) - [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022) - [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017) - [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020) - [Person Transfer GAN to Bridge Domain Gap for Person Re-Identification](https://mlsanity.com/paper/1711.08565/) (2017) - [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017) - [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021) - [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021) - [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019) - [Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro](https://mlsanity.com/paper/1701.07717/) (2017) - [An Empirical Study of Training End-to-End Vision-and-Language Transformers](https://mlsanity.com/paper/2111.02387/) (2021) - [CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels](https://mlsanity.com/paper/2211.13977/) (2022) - [Zero-Shot Composed Image Retrieval with Textual Inversion](https://mlsanity.com/paper/2303.15247/) (2023) - [FastReID: A Pytorch Toolbox for General Instance Re-identification](https://mlsanity.com/paper/2006.02631/) (2020) - [Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2302.03084/) (2023) - [Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformer](https://mlsanity.com/paper/2106.04095/) (2021) - [Identity-Guided Human Semantic Parsing for Person Re-Identification](https://mlsanity.com/paper/2007.13467/) (2020) - [Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification](https://mlsanity.com/paper/2108.08728/) (2021) - [High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification](https://mlsanity.com/paper/2003.08177/) (2020) - [Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification](https://mlsanity.com/paper/2205.02151/) (2022) - [Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark](https://mlsanity.com/paper/2306.02898/) (2023) - [A Pose-Sensitive Embedding for Person Re-Identification with Expanded Cross Neighborhood Re-Ranking](https://mlsanity.com/paper/1711.10378/) (2017) - [Deeply-Learned Part-Aligned Representations for Person Re-Identification](https://mlsanity.com/paper/1707.07256/) (2017) - [Learning Deep Context-aware Features over Body and Latent Parts for Person Re-identification](https://mlsanity.com/paper/1710.06555/) (2017) - [Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2309.16137/) (2023) - [Auto-ReID: Searching for a Part-aware ConvNet for Person Re-Identification](https://mlsanity.com/paper/1903.09776/) (2019) - [AAformer: Auto-Aligned Transformer for Person Re-Identification](https://mlsanity.com/paper/2104.00921/) (2021) ## Built on by (most cited first, 4 of 4) - [IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identification](https://mlsanity.com/paper/2503.10324/) (2025) - [Beyond Pedestrians: Caption-Guided CLIP Framework for High-Difficulty Video-based Person Re-Identification](https://mlsanity.com/paper/2604.07740/) (2026) - [From Global to Local: Rethinking CLIP Feature Aggregation for Person Re-Identification](https://mlsanity.com/paper/2604.22190/) (2026) - [A Generative AI Integrated Multimodal Framework for Low-Latency Multi-Camera Person Re-Identification](https://mlsanity.com/paper/2609.14419/) (2026) ## Abstract (arXiv) Person re-identification (ReID) has recently benefited from large pretrained vision-language models such as Contrastive Language-Image Pre-Training (CLIP). However, the absence of concrete descriptions necessitates the use of implicit text embeddings, which demand complicated and inefficient training strategies. To address this issue, we first propose one straightforward solution by leveraging existing image captioning models to generate pseudo captions for person images, and thereby boost person re-identification with large vision language models. Using models like the Large Language and Vision Assistant (LLAVA), we generate high-quality captions based on fixed templates that capture key semantic attributes such as gender, clothing, and age. By augmenting ReID training sets from uni-modality (image) to bi-modality (image and text), we introduce CLIP-SCGI, a simple yet effective framework that leverages synthesized captions to guide the learning of discriminative and robust representations. Built on CLIP, CLIP-SCGI fuses image and text embeddings through two modules to enhance the training process. To address quality issues in generated captions, we introduce a caption-guided inversion module that captures semantic attributes from images by converting relevant visual information into pseudo-word tokens based on the descriptions. This approach helps the model better capture key information and focus on relevant regions. The extracted features are then utilized in a cross-modal fusion module, guiding the model to focus on regions semantically consistent with the caption, thereby facilitating the optimization of the visual encoder to extract discriminative and robust representations. Extensive experiments on four popular ReID benchmarks demonstrate that CLIP-SCGI outperforms the state-of-the-art by a significant margin.