# Exploiting the Textual Potential from Vision-Language Pre-training for Text-based Person Search

arXiv: https://arxiv.org/abs/2303.04497 · HTML: https://mlsanity.com/paper/2303.04497/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 35 of its 43 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks](https://mlsanity.com/paper/2208.10442/) (2022)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Person Transfer GAN to Bridge Domain Gap for Person Re-Identification](https://mlsanity.com/paper/1711.08565/) (2017)
- [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018)
- [How Much Can CLIP Benefit Vision-and-Language Tasks?](https://mlsanity.com/paper/2107.06383/) (2021)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [Person Search with Natural Language Description](https://mlsanity.com/paper/1702.05729/) (2017)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment](https://mlsanity.com/paper/2203.07190/) (2022)
- [Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification](https://mlsanity.com/paper/2107.12666/) (2021)
- [DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrieval](https://mlsanity.com/paper/2109.05534/) (2021)
- [CLIP-Driven Fine-grained Text-Image Person Re-identification](https://mlsanity.com/paper/2210.10276/) (2022)
- [ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language](https://mlsanity.com/paper/2005.07327/) (2020)
- [Adversarial Representation Learning for Text-to-Image Matching](https://mlsanity.com/paper/1908.10534/) (2019)
- [See Finer, See More: Implicit Modality Alignment for Text-based Person Retrieval](https://mlsanity.com/paper/2208.08608/) (2022)
- [Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments](https://mlsanity.com/paper/1906.09610/) (2019)
- [Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search](https://mlsanity.com/paper/2101.03036/) (2021)
- [Learning Granularity-Unified Representations for Text-to-Image Person Re-identification](https://mlsanity.com/paper/2207.07802/) (2022)
- [TIPCB: A Simple but Effective Part-based Convolutional Baseline for Text-based Person Search](https://mlsanity.com/paper/2105.11628/) (2021)
- [Text-Based Person Search with Limited Data](https://mlsanity.com/paper/2110.10807/) (2021)
- [Learning Semantic-Aligned Feature Representation for Text-based Person Search](https://mlsanity.com/paper/2112.06714/) (2021)

## Built on by (most cited first, 7 of 7)

- [An Empirical Study of CLIP for Text-based Person Search](https://mlsanity.com/paper/2308.10045/) (2023)
- [Transformer for Object Re-Identification: A Survey](https://mlsanity.com/paper/2401.06960/) (2024)
- [From Attributes to Natural Language: A Survey and Foresight on Text-based Person Re-identification](https://mlsanity.com/paper/2408.00096/) (2024)
- [Enhancing Visual Representation for Text-based Person Searching](https://mlsanity.com/paper/2412.20646/) (2024)
- [Background Matters Too: A Language-Enhanced Adversarial Framework for Person Re-Identification](https://mlsanity.com/paper/2509.03032/) (2025)
- [Cross-Resolution Semantic Transfer for Robust Text-to-Image Retrieval in Low-Resolution Surveillance](https://mlsanity.com/paper/2606.30458/) (2026)
- [Blurring Modal Boundaries: A Unified Survey from Single- to Multi-Modal Person Re-ldentification](https://mlsanity.com/paper/2607.14821/) (2026)

## Abstract (arXiv)

Text-based Person Search (TPS), is targeted on retrieving pedestrians to match text descriptions instead of query images. Recent Vision-Language Pre-training (VLP) models can bring transferable knowledge to downstream TPS tasks, resulting in more efficient performance gains. However, existing TPS methods improved by VLP only utilize pre-trained visual encoders, neglecting the corresponding textual representation and breaking the significant modality alignment learned from large-scale pre-training. In this paper, we explore the full utilization of textual potential from VLP in TPS tasks. We build on the proposed VLP-TPS baseline model, which is the first TPS model with both pre-trained modalities. We propose the Multi-Integrity Description Constraints (MIDC) to enhance the robustness of the textual modality by incorporating different components of fine-grained corpus during training. Inspired by the prompt approach for zero-shot classification with VLP models, we propose the Dynamic Attribute Prompt (DAP) to provide a unified corpus of fine-grained attributes as language hints for the image modality. Extensive experiments show that our proposed TPS framework achieves state-of-the-art performance, exceeding the previous best method by a margin.