# Matching Images and Text with Multi-modal Tensor Fusion and Re-ranking

arXiv: https://arxiv.org/abs/1908.04011 · HTML: https://mlsanity.com/paper/1908.04011/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2019.

## Built on by (most cited first, 20 of 20)

- [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023)
- [DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrieval](https://mlsanity.com/paper/2109.05534/) (2021)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [Exploring a Fine-Grained Multiscale Method for Cross-Modal Remote Sensing Image Retrieval](https://mlsanity.com/paper/2204.09868/) (2022)
- [Remote Sensing Cross-Modal Text-Image Retrieval Based on Global and Local Information](https://mlsanity.com/paper/2204.09860/) (2022)
- [Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval](https://mlsanity.com/paper/2308.12509/) (2023)
- [Learning to Evaluate Performance of Multi-modal Semantic Localization](https://mlsanity.com/paper/2209.06515/) (2022)
- [Learnable Pillar-based Re-ranking for Image-Text Retrieval](https://mlsanity.com/paper/2304.12570/) (2023)
- [CFIR: Fast and Effective Long-Text To Image Retrieval for Large Corpora](https://mlsanity.com/paper/2402.15276/) (2024)
- [DocumentCLIP: Linking Figures and Main Body Text in Reflowed Documents](https://mlsanity.com/paper/2306.06306/) (2023)
- [Scale-Semantic Joint Decoupling Network for Image-text Retrieval in Remote Sensing](https://mlsanity.com/paper/2212.05752/) (2022)
- [Transcending Fusion: A Multi-Scale Alignment Method for Remote Sensing Image-Text Retrieval](https://mlsanity.com/paper/2405.18959/) (2024)
- [GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learning](https://mlsanity.com/paper/2410.15266/) (2024)
- [Diffusion-based Multi-modal Synergy Interest Network for Click-through Rate Prediction](https://mlsanity.com/paper/2508.21460/) (2025)
- [Dynamic Visual Semantic Sub-Embeddings and Fast Re-Ranking](https://mlsanity.com/paper/2309.08154/) (2023)
- [Cross-Modal Pre-Aligned Method with Global and Local Information for Remote-Sensing Image and Text Retrieval](https://mlsanity.com/paper/2411.14704/) (2024)
- [iEBAKER: Improved Remote Sensing Image-Text Retrieval Framework via Eliminate Before Align and Keyword Explicit Reasoning](https://mlsanity.com/paper/2504.05644/) (2025)
- [Representation Discrepancy Bridging Method for Remote Sensing Image-Text Retrieval](https://mlsanity.com/paper/2505.16756/) (2025)
- [FLEX-CLIP: Feature-Level GEneration Network Enhanced CLIP for X-shot Cross-modal Retrieval](https://mlsanity.com/paper/2411.17454/) (2024)
- [AlignJEPA: Predictive Vision-Language Alignment for Remote Sensing Foundation Models](https://mlsanity.com/paper/2608.15456/) (2026)

## Abstract (arXiv)

A major challenge in matching images and text is that they have intrinsically different data distributions and feature representations. Most existing approaches are based either on embedding or classification, the first one mapping image and text instances into a common embedding space for distance measuring, and the second one regarding image-text matching as a binary classification problem. Neither of these approaches can, however, balance the matching accuracy and model complexity well. We propose a novel framework that achieves remarkable matching performance with acceptable model complexity. Specifically, in the training stage, we propose a novel Multi-modal Tensor Fusion Network (MTFN) to explicitly learn an accurate image-text similarity function with rank-based tensor fusion rather than seeking a common embedding space for each image-text instance. Then, during testing, we deploy a generic Cross-modal Re-ranking (RR) scheme for refinement without requiring additional training procedure. Extensive experiments on two datasets demonstrate that our MTFN-RR consistently achieves the state-of-the-art matching performance with much less time complexity. The implementation code is available at https://github.com/Wangt-CN/MTFN-RR-PyTorch-Code.