# Prototype-Guided Text-based Person Search based on Rich Chinese Descriptions

arXiv: https://arxiv.org/abs/2312.14834 · HTML: https://mlsanity.com/paper/2312.14834/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 54 references that are in the map.
- First posted on arXiv 22 Dec 2023.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Joint Detection and Identification Feature Learning for Person Search](https://mlsanity.com/paper/1604.01850/) (2016)
- [Learning Deep Representations of Fine-grained Visual Descriptions](https://mlsanity.com/paper/1605.05395/) (2016)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [Person Re-identification in the Wild](https://mlsanity.com/paper/1604.02531/) (2016)
- [Person Search with Natural Language Description](https://mlsanity.com/paper/1702.05729/) (2017)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [Cross-Batch Memory for Embedding Learning](https://mlsanity.com/paper/1912.06798/) (2019)
- [Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval](https://mlsanity.com/paper/2303.12501/) (2023)
- [Person Re-identification by Contour Sketch under Moderate Clothing Change](https://mlsanity.com/paper/2002.02295/) (2020)
- [ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language](https://mlsanity.com/paper/2005.07327/) (2020)
- [Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/1912.01230/) (2019)
- [Adversarial Representation Learning for Text-to-Image Matching](https://mlsanity.com/paper/1908.10534/) (2019)
- [Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments](https://mlsanity.com/paper/1906.09610/) (2019)
- [Learning Context Graph for Person Search](https://mlsanity.com/paper/1904.01830/) (2019)
- [Person Search via A Mask-Guided Two-Stream CNN Model](https://mlsanity.com/paper/1807.08107/) (2018)
- [Anchor-Free Person Search](https://mlsanity.com/paper/2103.11617/) (2021)
- [Re-ID Driven Localization Refinement for Person Search](https://mlsanity.com/paper/1909.08580/) (2019)
- [Query-guided End-to-End Person Search](https://mlsanity.com/paper/1905.01203/) (2019)
- [Person Search by Multi-Scale Matching](https://mlsanity.com/paper/1807.08582/) (2018)
- [Neural Person Search Machines](https://mlsanity.com/paper/1707.06777/) (2017)

## Abstract (arXiv)

Text-based person search aims to simultaneously localize and identify the target person based on query text from uncropped scene images, which can be regarded as the unified task of person detection and text-based person retrieval task. In this work, we propose a large-scale benchmark dataset named PRW-TPS-CN based on the widely used person search dataset PRW. Our dataset contains 47,102 sentences, which means there is quite more information than existing dataset. These texts precisely describe the person images from top to bottom, which in line with the natural description order. We also provide both Chinese and English descriptions in our dataset for more comprehensive evaluation. These characteristics make our dataset more applicable. To alleviate the inconsistency between person detection and text-based person retrieval, we take advantage of the rich texts in PRW-TPS-CN dataset. We propose to aggregate multiple texts as text prototypes to maintain the prominent text features of a person, which can better reflect the whole character of a person. The overall prototypes lead to generating the image attention map to eliminate the detection misalignment causing the decrease of text-based person retrieval. Thus, the inconsistency between person detection and text-based person retrieval is largely alleviated. We conduct extensive experiments on the PRW-TPS-CN dataset. The experimental results show the PRW-TPS-CN dataset's effectiveness and the state-of-the-art performance of our approach.