# Identity-Aware Textual-Visual Matching with Latent Co-attention

arXiv: https://arxiv.org/abs/1708.01988 · HTML: https://mlsanity.com/paper/1708.01988/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2017.

## Built on by (most cited first, 30 of 54)

- [Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval](https://mlsanity.com/paper/2303.12501/) (2023)
- [Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification](https://mlsanity.com/paper/2107.12666/) (2021)
- [DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrieval](https://mlsanity.com/paper/2109.05534/) (2021)
- [Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark](https://mlsanity.com/paper/2306.02898/) (2023)
- [See Finer, See More: Implicit Modality Alignment for Text-based Person Retrieval](https://mlsanity.com/paper/2208.08608/) (2022)
- [Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search](https://mlsanity.com/paper/2101.03036/) (2021)
- [Learning Granularity-Unified Representations for Text-to-Image Person Re-identification](https://mlsanity.com/paper/2207.07802/) (2022)
- [TIPCB: A Simple but Effective Part-based Convolutional Baseline for Text-based Person Search](https://mlsanity.com/paper/2105.11628/) (2021)
- [Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering](https://mlsanity.com/paper/2104.14085/) (2021)
- [RaSa: Relation and Sensitivity Aware Representation Learning for Text-based Person Search](https://mlsanity.com/paper/2305.13653/) (2023)
- [Learning Semantic-Aligned Feature Representation for Text-based Person Search](https://mlsanity.com/paper/2112.06714/) (2021)
- [CAIBC: Capturing All-round Information Beyond Color for Text-based Person Retrieval](https://mlsanity.com/paper/2209.05773/) (2022)
- [Image-Specific Information Suppression and Implicit Local Alignment for Text-based Person Search](https://mlsanity.com/paper/2208.14365/) (2022)
- [Look Before You Leap: Improving Text-based Person Retrieval by Learning A Consistent Cross-modal Common Manifold](https://mlsanity.com/paper/2209.06209/) (2022)
- [An Empirical Study of CLIP for Text-based Person Search](https://mlsanity.com/paper/2308.10045/) (2023)
- [AXM-Net: Implicit Cross-Modal Feature Alignment for Person Re-identification](https://mlsanity.com/paper/2101.08238/) (2021)
- [Unified Pre-training with Pseudo Texts for Text-To-Image Person Re-identification](https://mlsanity.com/paper/2309.01420/) (2023)
- [Text-based Person Search without Parallel Image-Text Data](https://mlsanity.com/paper/2305.12964/) (2023)
- [PoseScript: Linking 3D Human Poses and Natural Language](https://mlsanity.com/paper/2210.11795/) (2022)
- [From Data Deluge to Data Curation: A Filtering-WoRA Paradigm for Efficient Text-based Person Search](https://mlsanity.com/paper/2404.10292/) (2024)
- [ASMR: Learning Attribute-Based Person Search with Adaptive Semantic Margin Regularizer](https://mlsanity.com/paper/2108.04533/) (2021)
- [Cross-Modal Retrieval for Motion and Text via DropTriple Loss](https://mlsanity.com/paper/2305.04195/) (2023)
- [VGSG: Vision-Guided Semantic-Group Network for Text-based Person Search](https://mlsanity.com/paper/2311.07514/) (2023)
- [UPAR: Unified Pedestrian Attribute Recognition and Person Retrieval](https://mlsanity.com/paper/2209.02522/) (2022)
- [Text-based Person Search in Full Images via Semantic-Driven Proposal Generation](https://mlsanity.com/paper/2109.12965/) (2021)
- [Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval](https://mlsanity.com/paper/2104.04991/) (2021)
- [Person Search Challenges and Solutions: A Survey](https://mlsanity.com/paper/2105.01605/) (2021)
- [Person Retrieval in Surveillance Using Textual Query: A Review](https://mlsanity.com/paper/2105.02414/) (2021)
- [Multi-modal Reference Learning for Fine-grained Text-to-Image Retrieval](https://mlsanity.com/paper/2504.07718/) (2025)
- [Masked Attribute Description Embedding for Cloth-Changing Person Re-identification](https://mlsanity.com/paper/2401.05646/) (2024)

## Abstract (arXiv)

Textual-visual matching aims at measuring similarities between sentence descriptions and images. Most existing methods tackle this problem without effectively utilizing identity-level annotations. In this paper, we propose an identity-aware two-stage framework for the textual-visual matching problem. Our stage-1 CNN-LSTM network learns to embed cross-modal features with a novel Cross-Modal Cross-Entropy (CMCE) loss. The stage-1 network is able to efficiently screen easy incorrect matchings and also provide initial training point for the stage-2 training. The stage-2 CNN-LSTM network refines the matching results with a latent co-attention mechanism. The spatial attention relates each word with corresponding image regions while the latent semantic attention aligns different sentence structures to make the matching results more robust to sentence structure variations. Extensive experiments on three datasets with identity-level annotations show that our framework outperforms state-of-the-art approaches by large margins.