# Look Before You Leap: Improving Text-based Person Retrieval by Learning A Consistent Cross-modal Common Manifold

arXiv: https://arxiv.org/abs/2209.06209 · HTML: https://mlsanity.com/paper/2209.06209/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Person Transfer GAN to Bridge Domain Gap for Person Re-Identification](https://mlsanity.com/paper/1711.08565/) (2017)
- [Learning Deep Representations of Fine-grained Visual Descriptions](https://mlsanity.com/paper/1605.05395/) (2016)
- [Person Search with Natural Language Description](https://mlsanity.com/paper/1702.05729/) (2017)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification](https://mlsanity.com/paper/2107.12666/) (2021)
- [Unsupervised Person Re-identification via Multi-label Classification](https://mlsanity.com/paper/2004.09228/) (2020)
- [DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrieval](https://mlsanity.com/paper/2109.05534/) (2021)
- [Interaction-and-Aggregation Network for Person Re-identification](https://mlsanity.com/paper/1907.08435/) (2019)
- [Unsupervised Person Re-identification: Clustering and Fine-tuning](https://mlsanity.com/paper/1705.10444/) (2017)
- [Adversarial Representation Learning for Text-to-Image Matching](https://mlsanity.com/paper/1908.10534/) (2019)
- [Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments](https://mlsanity.com/paper/1906.09610/) (2019)
- [Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-Identification](https://mlsanity.com/paper/1803.09786/) (2018)
- [Identity-Aware Textual-Visual Matching with Latent Co-attention](https://mlsanity.com/paper/1708.01988/) (2017)
- [Pose-Guided Multi-Granularity Attention Network for Text-Based Person Search](https://mlsanity.com/paper/1809.08440/) (2018)
- [Second-order Non-local Attention Networks for Person Re-identification](https://mlsanity.com/paper/1909.00295/) (2019)
- [Multi-task Mid-level Feature Alignment Network for Unsupervised Cross-Dataset Person Re-Identification](https://mlsanity.com/paper/1807.01440/) (2018)
- [Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association](https://mlsanity.com/paper/1808.01571/) (2018)
- [Global Distance-distributions Separation for Unsupervised Person Re-identification](https://mlsanity.com/paper/2006.00752/) (2020)
- [Cross-Modal Food Retrieval: Learning a Joint Embedding of Food Images and Recipes with Semantic Consistency and Attention Mechanism](https://mlsanity.com/paper/2003.03955/) (2020)

## Built on by (most cited first, 30 of 38)

- [Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval](https://mlsanity.com/paper/2303.12501/) (2023)
- [CLIP-Driven Fine-grained Text-Image Person Re-identification](https://mlsanity.com/paper/2210.10276/) (2022)
- [Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark](https://mlsanity.com/paper/2306.02898/) (2023)
- [Noisy-Correspondence Learning for Text-to-Image Person Re-identification](https://mlsanity.com/paper/2308.09911/) (2023)
- [Chat-Driven Text Generation and Interaction for Person Retrieval](https://mlsanity.com/paper/2509.12662/) (2025)
- [HAP: Structure-Aware Masked Image Modeling for Human-Centric Perception](https://mlsanity.com/paper/2310.20695/) (2023)
- [UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity](https://mlsanity.com/paper/2312.03441/) (2023)
- [Unified Pre-training with Pseudo Texts for Text-To-Image Person Re-identification](https://mlsanity.com/paper/2309.01420/) (2023)
- [Text-based Person Search without Parallel Image-Text Data](https://mlsanity.com/paper/2305.12964/) (2023)
- [From Data Deluge to Data Curation: A Filtering-WoRA Paradigm for Efficient Text-based Person Search](https://mlsanity.com/paper/2404.10292/) (2024)
- [VGSG: Vision-Guided Semantic-Group Network for Text-based Person Search](https://mlsanity.com/paper/2311.07514/) (2023)
- [Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Search](https://mlsanity.com/paper/2411.17776/) (2024)
- [BiLMa: Bidirectional Local-Matching for Text-based Person Re-identification](https://mlsanity.com/paper/2309.04675/) (2023)
- [Modeling Thousands of Human Annotators for Generalizable Text-to-Image Person Re-identification](https://mlsanity.com/paper/2503.09962/) (2025)
- [Exploiting the Textual Potential from Vision-Language Pre-training for Text-based Person Search](https://mlsanity.com/paper/2303.04497/) (2023)
- [Learning Comprehensive Representations with Richer Self for Text-to-Image Person Re-Identification](https://mlsanity.com/paper/2310.11210/) (2023)
- [DGNS: Deformable Gaussian Splatting and Dynamic Neural Surface for Monocular Dynamic 3D Reconstruction](https://mlsanity.com/paper/2412.03910/) (2024)
- [Multi-modal Reference Learning for Fine-grained Text-to-Image Retrieval](https://mlsanity.com/paper/2504.07718/) (2025)
- [CAMeL: Cross-modality Adaptive Meta-Learning for Text-based Person Retrieval](https://mlsanity.com/paper/2504.18782/) (2025)
- [Cross-Modal Adaptive Dual Association for Text-to-Image Person Retrieval](https://mlsanity.com/paper/2312.01745/) (2023)
- [Beat: Bi-directional One-to-Many Embedding Alignment for Text-based Person Retrieval](https://mlsanity.com/paper/2406.05620/) (2024)
- [From Attributes to Natural Language: A Survey and Foresight on Text-based Person Re-identification](https://mlsanity.com/paper/2408.00096/) (2024)
- [Gradient-Attention Guided Dual-Masking Synergetic Framework for Robust Text-based Person Retrieval](https://mlsanity.com/paper/2509.09118/) (2025)
- [Bridging the Pose-Semantic Gap: A Cascade Framework for Text-Based Person Anomaly Search](https://mlsanity.com/paper/2604.23282/) (2026)
- [Enhancing Visual Representation for Text-based Person Searching](https://mlsanity.com/paper/2412.20646/) (2024)
- [Minimizing the Pretraining Gap: Domain-aligned Text-Based Person Retrieval](https://mlsanity.com/paper/2507.10195/) (2025)
- [FlexiReID: Adaptive Mixture of Expert for Multi-Modal Person Re-Identification](https://mlsanity.com/paper/2510.15595/) (2025)
- [Text-guided Image Restoration and Semantic Enhancement for Text-to-Image Person Retrieval](https://mlsanity.com/paper/2307.09059/) (2023)
- [Progressive Feature Mining and External Knowledge-Assisted Text-Pedestrian Image Retrieval](https://mlsanity.com/paper/2308.11994/) (2023)
- [Prompt Decoupling for Text-to-Image Person Re-identification](https://mlsanity.com/paper/2401.02173/) (2024)

## Abstract (arXiv)

The core problem of text-based person retrieval is how to bridge the heterogeneous gap between multi-modal data. Many previous approaches contrive to learning a latent common manifold mapping paradigm following a cross-modal distribution consensus prediction (CDCP) manner. When mapping features from distribution of one certain modality into the common manifold, feature distribution of the opposite modality is completely invisible. That is to say, how to achieve a cross-modal distribution consensus so as to embed and align the multi-modal features in a constructed cross-modal common manifold all depends on the experience of the model itself, instead of the actual situation. With such methods, it is inevitable that the multi-modal data can not be well aligned in the common manifold, which finally leads to a sub-optimal retrieval performance. To overcome this CDCP dilemma, we propose a novel algorithm termed LBUL to learn a Consistent Cross-modal Common Manifold (C3M) for text-based person retrieval. The core idea of our method, just as a Chinese saying goes, is to `textitsan si er hou xing', namely, to Look Before yoU Leap (LBUL). The common manifold mapping mechanism of LBUL contains a looking step and a leaping step. Compared to CDCP-based methods, LBUL considers distribution characteristics of both the visual and textual modalities before embedding data from one certain modality into C3M to achieve a more solid cross-modal distribution consensus, and hence achieve a superior retrieval accuracy. We evaluate our proposed method on two text-based person retrieval datasets CUHK-PEDES and RSTPReid. Experimental results demonstrate that the proposed LBUL outperforms previous methods and achieves the state-of-the-art performance.