UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Universal multimodal retrieval aims to support diverse instruction-aware retrieval tasks, demanding both efficient corpus-scale matching and fine-grained semantic reasoning. Recent MLLM-based embedding methods typically derive representations from hidden states, while Chain-of-Thought (CoT) reasoning is emerging as a promising strategy for embedding enhancement by encoding intermediate semantic evidence into the representation space. However, existing CoT methods typically use item-wise reasoning over queries and candidates in isolation, providing no explicit evidence to distinguish a positive from a semantically confusable hard negative. Moreover, contrastive embeddings capture global similarity but struggle with meta-tasks requiring answer verification, category judgment or fine-grained reasoning. In this paper, we propose UMER, a Unified Multimodal Embedding and Ranking framework for universal multimodal retrieval. UMER replaces item-wise reflection with Pair-Aware Discriminative Reasoning, which compares query--candidate pairs to identify instruction-relevant matching and discrepancy evidence. UMER jointly learns contrastive embeddings for efficient global matching and discriminative ranking for explicit pairwise relevance judgment within a single MLLM. A complementary mutual distillation strategy further transfers reliable pairwise preferences between the embedding and ranking functions. On the MMEB-V2 benchmark, UMER achieves state-of-the-art performance under comparable experimental settings while supporting budget-adjustable inference.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 25 references that are in the map. First posted on arXiv 19 Aug 2026.
Builds on
15 of 25 references.
Closest ancestors · 7
Same direction, newest first
- 2026LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck11 Jun
- 2026Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings24 Apr
- 2026MMEmb-R1: Reasoning-Enhanced Multimodal Embedding with Pair-Aware Selection and Adaptive Control7 Apr
- 2026PLUME: Latent Reasoning Based Universal Multimodal Embedding2 Apr
- 2026Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddings14 Feb
- 2025UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings1 Nov
- 2025Think Then Embed: Generative Context Improves Multimodal Embedding6 Oct
Other references · 8
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
- 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
- 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 126
- 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 116
- 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
- 2024MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Trainingcited by 50
- 2026Bottleneck Tokens for Unified Multimodal Retrievalcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval
- 2025U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMs
- 2026TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrieval
- 2025Reason to Contrast: A Cascaded Multimodal Retrieval Framework
- 2026Reasoning-Augmented Representations for Multimodal Retrieval
- 2026Is Reasoning Always Useful? Rethinking Reasoning Utility in Universal Multimodal Embeddings
- 2025Reasoning Guided Embeddings: Leveraging MLLM Reasoning for Improved Multimodal Retrieval
- 2025ReMatch: Boosting Representation through Matching for Multimodal Retrieval