# DEMO: A Statistical Perspective for Efficient Image-Text Matching

arXiv: https://arxiv.org/abs/2405.11496 · HTML: https://mlsanity.com/paper/2405.11496/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 56 references that are in the map.
- First posted on arXiv 19 May 2024.

## Builds on (most cited first, 17 of 17)

- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval](https://mlsanity.com/paper/2003.03772/) (2020)
- [Image-text Retrieval: A Survey on Recent Research and Development](https://mlsanity.com/paper/2203.14713/) (2022)
- [Deep Cross-Modal Hashing](https://mlsanity.com/paper/1602.02255/) (2016)
- [Focus Your Attention: A Bidirectional Focal Attention Network for Image-Text Matching](https://mlsanity.com/paper/1909.11416/) (2019)
- [CO2: Consistent Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2010.02217/) (2020)
- [Unsupervised Generative Adversarial Cross-modal Hashing](https://mlsanity.com/paper/1712.00358/) (2017)
- [Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing](https://mlsanity.com/paper/2012.13538/) (2020)
- [Accelerating Code Search with Deep Hashing and Code Classification](https://mlsanity.com/paper/2203.15287/) (2022)
- [A Strong and Robust Baseline for Text-Image Matching](https://mlsanity.com/paper/1906.01205/) (2019)
- [MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal Retrieval](https://mlsanity.com/paper/1805.01963/) (2018)
- [CIMON: Towards High-quality Hash Codes](https://mlsanity.com/paper/2010.07804/) (2020)
- [Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis](https://mlsanity.com/paper/2305.16166/) (2023)
- [Unsupervised Contrastive Hashing for Cross-Modal Retrieval in Remote Sensing](https://mlsanity.com/paper/2204.08707/) (2022)
- [UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding](https://mlsanity.com/paper/2307.00862/) (2023)

## Abstract (arXiv)

Image-text matching has been a long-standing problem, which seeks to connect vision and language through semantic understanding. Due to the capability to manage large-scale raw data, unsupervised hashing-based approaches have gained prominence recently. They typically construct a semantic similarity structure using the natural distance, which subsequently provides guidance to the model optimization process. However, the similarity structure could be biased at the boundaries of semantic distributions, causing error accumulation during sequential optimization. To tackle this, we introduce a novel hashing approach termed Distribution-based Structure Mining with Consistency Learning (DEMO) for efficient image-text matching. From a statistical view, DEMO characterizes each image using multiple augmented views, which are considered as samples drawn from its intrinsic semantic distribution. Then, we employ a non-parametric distribution divergence to ensure a robust and precise similarity structure. In addition, we introduce collaborative consistency learning which not only preserves the similarity structure in the Hamming space but also encourages consistency between retrieval distribution from different directions in a self-supervised manner. Through extensive experiments on three benchmark image-text matching datasets, we demonstrate that DEMO achieves superior performance compared with many state-of-the-art methods.