# Dual-Path Convolutional Image-Text Embeddings with Instance Loss

arXiv: https://arxiv.org/abs/1711.05535 · HTML: https://mlsanity.com/paper/1711.05535/

## Key facts

- Cited by 145 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2017.

## Built on by (most cited first, 30 of 145)

- [Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval](https://mlsanity.com/paper/2303.12501/) (2023)
- [VinVL: Revisiting Visual Representations in Vision-Language Models](https://mlsanity.com/paper/2101.00529/) (2021)
- [Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification](https://mlsanity.com/paper/2107.12666/) (2021)
- [DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrieval](https://mlsanity.com/paper/2109.05534/) (2021)
- [CLIP-Driven Fine-grained Text-Image Person Re-identification](https://mlsanity.com/paper/2210.10276/) (2022)
- [Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark](https://mlsanity.com/paper/2306.02898/) (2023)
- [Multi-Modal Knowledge Graph Construction and Application: A Survey](https://mlsanity.com/paper/2202.05786/) (2022)
- [See Finer, See More: Implicit Modality Alignment for Text-based Person Retrieval](https://mlsanity.com/paper/2208.08608/) (2022)
- [Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search](https://mlsanity.com/paper/2101.03036/) (2021)
- [Learning Granularity-Unified Representations for Text-to-Image Person Re-identification](https://mlsanity.com/paper/2207.07802/) (2022)
- [TIPCB: A Simple but Effective Part-based Convolutional Baseline for Text-based Person Search](https://mlsanity.com/paper/2105.11628/) (2021)
- [A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-Localization](https://mlsanity.com/paper/2201.09206/) (2022)
- [Noisy-Correspondence Learning for Text-to-Image Person Re-identification](https://mlsanity.com/paper/2308.09911/) (2023)
- [Image-text Retrieval: A Survey on Recent Research and Development](https://mlsanity.com/paper/2203.14713/) (2022)
- [Text-Based Person Search with Limited Data](https://mlsanity.com/paper/2110.10807/) (2021)
- [RaSa: Relation and Sensitivity Aware Representation Learning for Text-based Person Search](https://mlsanity.com/paper/2305.13653/) (2023)
- [PLIP: Language-Image Pre-training for Person Representation Learning](https://mlsanity.com/paper/2305.08386/) (2023)
- [Vision-Based UAV Self-Positioning in Low-Altitude Urban Environments](https://mlsanity.com/paper/2201.09201/) (2022)
- [Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions](https://mlsanity.com/paper/2306.07520/) (2023)
- [Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization](https://mlsanity.com/paper/2211.07394/) (2022)
- [Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching](https://mlsanity.com/paper/2311.12751/) (2023)
- [Learning Semantic-Aligned Feature Representation for Text-based Person Search](https://mlsanity.com/paper/2112.06714/) (2021)
- [CAIBC: Capturing All-round Information Beyond Color for Text-based Person Retrieval](https://mlsanity.com/paper/2209.05773/) (2022)
- [Image-Specific Information Suppression and Implicit Local Alignment for Text-based Person Search](https://mlsanity.com/paper/2208.14365/) (2022)
- [Look Before You Leap: Improving Text-based Person Retrieval by Learning A Consistent Cross-modal Common Manifold](https://mlsanity.com/paper/2209.06209/) (2022)
- [An Empirical Study of CLIP for Text-based Person Search](https://mlsanity.com/paper/2308.10045/) (2023)
- [Where am I? Cross-View Geo-localization with Natural Language Descriptions](https://mlsanity.com/paper/2412.17007/) (2024)
- [Multiple-environment Self-adaptive Network for Aerial-view Geo-localization](https://mlsanity.com/paper/2204.08381/) (2022)
- [AXM-Net: Implicit Cross-Modal Feature Alignment for Person Re-identification](https://mlsanity.com/paper/2101.08238/) (2021)
- [Chat-Driven Text Generation and Interaction for Person Retrieval](https://mlsanity.com/paper/2509.12662/) (2025)

## Abstract (arXiv)

Matching images and sentences demands a fine understanding of both modalities. In this paper, we propose a new system to discriminatively embed the image and text to a shared visual-textual space. In this field, most existing works apply the ranking loss to pull the positive image / text pairs close and push the negative pairs apart from each other. However, directly deploying the ranking loss is hard for network learning, since it starts from the two heterogeneous features to build inter-modal relationship. To address this problem, we propose the instance loss which explicitly considers the intra-modal data distribution. It is based on an unsupervised assumption that each image / text group can be viewed as a class. So the network can learn the fine granularity from every image/text group. The experiment shows that the instance loss offers better weight initialization for the ranking loss, so that more discriminative embeddings can be learned. Besides, existing works usually apply the off-the-shelf features, i.e., word2vec and fixed visual feature. So in a minor contribution, this paper constructs an end-to-end dual-path convolutional network to learn the image and text representations. End-to-end learning allows the system to directly learn from the data and fully utilize the supervision. On two generic retrieval datasets (Flickr30k and MSCOCO), experiments demonstrate that our method yields competitive accuracy compared to state-of-the-art methods. Moreover, in language based person retrieval, we improve the state of the art by a large margin. The code has been made publicly available.