# Universal Weighting Metric Learning for Cross-Modal Matching

arXiv: https://arxiv.org/abs/2010.03403 · HTML: https://mlsanity.com/paper/2010.03403/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 30 of 34)

- [Improving Cross-Modal Retrieval with Set of Diverse Embeddings](https://mlsanity.com/paper/2211.16761/) (2022)
- [HANet: Hierarchical Alignment Networks for Video-Text Retrieval](https://mlsanity.com/paper/2107.12059/) (2021)
- [CODER: Coupled Diversity-Sensitive Momentum Contrastive Learning for Image-Text Retrieval](https://mlsanity.com/paper/2208.09843/) (2022)
- [Telling the What while Pointing to the Where: Multimodal Queries for Image Retrieval](https://mlsanity.com/paper/2102.04980/) (2021)
- [Semantic Invariant Multi-view Clustering with Fully Incomplete Information](https://mlsanity.com/paper/2305.12743/) (2023)
- [Continual learning in cross-modal retrieval](https://mlsanity.com/paper/2104.06806/) (2021)
- [Revisiting Deep Audio-Text Retrieval Through the Lens of Transportation](https://mlsanity.com/paper/2405.10084/) (2024)
- [ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval](https://mlsanity.com/paper/2502.15682/) (2025)
- [One-shot Scene Graph Generation](https://mlsanity.com/paper/2202.10824/) (2022)
- [Aligning Information Capacity Between Vision and Language via Dense-to-Sparse Feature Distillation for Image-Text Matching](https://mlsanity.com/paper/2503.14953/) (2025)
- [GENIUS: A Generative Framework for Universal Multimodal Search](https://mlsanity.com/paper/2503.19868/) (2025)
- [Deep Metric Learning for Computer Vision: A Brief Overview](https://mlsanity.com/paper/2312.10046/) (2023)
- [Deep Boosting Learning: A Brand-new Cooperative Approach for Image-Text Matching](https://mlsanity.com/paper/2404.18114/) (2024)
- [GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learning](https://mlsanity.com/paper/2410.15266/) (2024)
- [ConTra: (Con)text (Tra)nsformer for Cross-Modal Video Retrieval](https://mlsanity.com/paper/2210.04341/) (2022)
- [Object-Aware Query Perturbation for Cross-Modal Image-Text Retrieval](https://mlsanity.com/paper/2407.12346/) (2024)
- [Semantic Enhanced Knowledge Graph for Large-Scale Zero-Shot Learning](https://mlsanity.com/paper/2212.13151/) (2022)
- [Selectively Hard Negative Mining for Alleviating Gradient Vanishing in Image-Text Matching](https://mlsanity.com/paper/2303.00181/) (2023)
- [SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval](https://mlsanity.com/paper/2501.08347/) (2025)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)
- [Unified Loss of Pair Similarity Optimization for Vision-Language Retrieval](https://mlsanity.com/paper/2209.13869/) (2022)
- [Image-Text Retrieval with Binary and Continuous Label Supervision](https://mlsanity.com/paper/2210.11319/) (2022)
- [Robust Calibrate Proxy Loss for Deep Metric Learning](https://mlsanity.com/paper/2304.09162/) (2023)
- [Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval](https://mlsanity.com/paper/2506.21538/) (2025)
- [Semantic Video Moments Retrieval at Scale: A New Task and a Baseline](https://mlsanity.com/paper/2210.08389/) (2022)
- [VMCML: Video and Music Matching via Cross-Modality Lifting](https://mlsanity.com/paper/2303.12379/) (2023)
- [Integrating Listwise Ranking into Pairwise-based Image-Text Retrieval](https://mlsanity.com/paper/2305.16566/) (2023)
- [InPK: Infusing Prior Knowledge into Prompt for Vision-Language Models](https://mlsanity.com/paper/2502.19777/) (2025)
- [Descriptive Image-Text Matching with Graded Contextual Similarity](https://mlsanity.com/paper/2505.09997/) (2025)
- [Multimodal Information Retrieval for Open World with Edit Distance Weak Supervision](https://mlsanity.com/paper/2506.20070/) (2025)

## Abstract (arXiv)

Cross-modal matching has been a highlighted research topic in both vision and language areas. Learning appropriate mining strategy to sample and weight informative pairs is crucial for the cross-modal matching performance. However, most existing metric learning methods are developed for unimodal matching, which is unsuitable for cross-modal matching on multimodal data with heterogeneous features. To address this problem, we propose a simple and interpretable universal weighting framework for cross-modal matching, which provides a tool to analyze the interpretability of various loss functions. Furthermore, we introduce a new polynomial loss under the universal weighting framework, which defines a weight function for the positive and negative informative pairs respectively. Experimental results on two image-text matching benchmarks and two video-text matching benchmarks validate the efficacy of the proposed method.