Data to 14 Oct 2026

Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models

arXiv 1711.06420 ↗First posted 17 Nov 2017cs.CVcited by 53

Not on the map. Why?

Abstract (arXiv)

Textual-visual cross-modal retrieval has been a hot research topic in both computer vision and natural language processing communities. Learning appropriate representations for multi-modal data is crucial for the cross-modal retrieval performance. Unlike existing image-text retrieval approaches that embed image-text pairs as single feature vectors in a common representational space, we propose to incorporate generative processes into the cross-modal feature embedding, through which we are able to learn not only the global abstract features but also the local grounded features. Extensive experiments show that our framework can well match images and sentences with complex content, and achieve the state-of-the-art cross-modal retrieval results on MSCOCO dataset.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 20; 2022: 15; 2023: 7; 2024: 5; 2025: 6.

53 papers.

Sort:

By area:

2021Multimodal Contrastive Training for Visual Representation Learningcited by 124
2021Similarity Reasoning and Filtration for Image-Text Matchingcited by 121
2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
2021Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identificationcited by 110
2021DOC2PPT: Automatic Presentation Slides Generation from Scientific Documentscited by 70
2021LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrievalcited by 44
2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 44
2023Improved Probabilistic Image-Text Representationscited by 40
2022ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCOcited by 37
2022Improving Cross-Modal Retrieval with Set of Diverse Embeddingscited by 33
2021Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learningcited by 28
2021Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Designcited by 23

38 more

2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
2022Imagination-Augmented Natural Language Understandingcited by 19
2023Vision-Language Dataset Distillationcited by 19
2021Learning Structural Representations for Recipe Generation and Food Retrievalcited by 17
2024Multimodal Learned Sparse Retrieval with Probabilistic Expansion Controlcited by 12
2022DR-GAN: Distribution Regularization for Text-to-Image Generationcited by 10
2023Plug-and-Play Regulators for Image-Text Matchingcited by 10
2025ELIP: Enhanced Visual-Language Foundation Models for Image Retrievalcited by 9
2021Discrete-continuous Action Space Policy Gradient-based Attention for Image-Text Matchingcited by 8
2021Cross-Modal Generative Augmentation for Visual Question Answeringcited by 7
2022On Distinctive Image Captioning via Comparing and Reweightingcited by 7
2025PHyCLIP: ℓ1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learningcited by 7
2021VLDeformer: Vision-Language Decomposed Transformer for Fast Cross-Modal Retrievalcited by 6
2022Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognitioncited by 6
2022LILE: Look In-Depth before Looking Elsewhere -- A Dual Attention Network using Transformers for Cross-Modal Information Retrieval in Histopathology Archivescited by 5
2023What Makes for Robust Multi-Modal Models in the Face of Missing Modalities?cited by 5
2024From Attributes to Natural Language: A Survey and Foresight on Text-based Person Re-identificationcited by 5
2025Aligning Information Capacity Between Vision and Language via Dense-to-Sparse Feature Distillation for Image-Text Matchingcited by 5
2021SwAMP: Swapped Assignment of Multi-Modal Pairs for Cross-Modal Retrievalcited by 4
2022HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrievalcited by 4
2022BOSS: Bottom-up Cross-modal Semantic Composition with Hybrid Counterfactual Training for Robust Content-based Image Retrievalcited by 4
2022See What You See: Self-supervised Cross-modal Retrieval of Visual Stimuli from Brain Activitycited by 4
2022Text-to-Image Generation via Implicit Visual Guidance and Hypernetworkcited by 4
2023ALR-GAN: Adaptive Layout Refinement for Text-to-Image Synthesiscited by 4
2024Deep Boosting Learning: A Brand-new Cooperative Approach for Image-Text Matchingcited by 4
2024GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learningcited by 4
2021Show, Write, and Retrieve: Entity-aware Article Generation and Retrievalcited by 3
2023Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Studycited by 3
2024Object-Aware Query Perturbation for Cross-Modal Image-Text Retrievalcited by 3
2021An Unsupervised Sampling Approach for Image-Sentence Matching Using Document-Level Structural Informationcited by 2
2021Survey of Visual-Semantic Embedding Methods for Zero-Shot Image Retrievalcited by 2
2022Two-stream Hierarchical Similarity Reasoning for Image-text Matchingcited by 2
2022A survey on Self Supervised learning approaches for improving Multimodal representation learningcited by 2
2025Hidden Bias in the Machine: Stereotypes in Text-to-Image Modelscited by 2
2021Macroscopic Control of Text Generation for Image Captioningcited by 1
2021A Universal Model for Cross Modality Mapping by Relational Reasoningcited by 1
2022Towards Practicality of Sketch-Based Visual Understandingcited by 1
2025Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrievalcited by 1

50 most cited of 53 All 53 (JSON)