mlsanityData to 7 Oct 2026

MMEmb-R1: Reasoning-Enhanced Multimodal Embedding with Pair-Aware Selection and Adaptive Control

arXiv 2604.06156 ↗First posted 7 Apr 2026cs.CV cs.AI cs.CLcited by 6

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

MLLMs have been successfully applied to multimodal embedding tasks, yet their generative reasoning capabilities remain underutilized. Directly incorporating chain-of-thought reasoning into embedding learning introduces two fundamental challenges. First, structural misalignment between instance-level reasoning and pairwise contrastive supervision may lead to shortcut behavior, where the model merely learns the superficial format of reasoning. Second, reasoning is not universally beneficial for embedding tasks. Enforcing reasoning for all inputs may introduce unnecessary computation and latency, and can even obscure salient semantic signals for simple cases. To address these issues, we propose MMEmb-R1, an adaptive reasoning-based multimodal embedding framework. We formulate reasoning as a latent variable and introduce pair-aware reasoning selection that employs counterfactual intervention to identify reasoning paths beneficial for query-target alignment. Furthermore, we adopt reinforcement learning to selectively invoke reasoning only when necessary. Experiments on the MMEB-V2 benchmark demonstrate that our model achieves a score of 71.2 with only 4B parameters, establishing a new state-of-the-art while significantly reducing reasoning overhead and inference latency.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 75 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

43 of 75 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddings14 Feb
  2. 2025UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings1 Nov
  3. 2025Think Then Embed: Generative Context Improves Multimodal Embedding6 Oct
  4. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
  5. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT1 May
  6. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  7. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  8. 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine11 Jul
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  10. 2022Emergent Abilities of Large Language Modelscited by 2,483
  11. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  12. 2024OpenAI o1 System Cardcited by 1,220
22 more
  1. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  2. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  3. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  4. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  5. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 190
  6. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  7. 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
  8. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 110
  9. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 82
  10. 2025VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videoscited by 76
  11. 2023Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyondcited by 50
  12. 2025UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learningcited by 41
  13. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 34
  14. 2025RzenEmbed: Towards Comprehensive Multimodal Retrievalcited by 28
  15. 2025MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddingscited by 20
  16. 2025SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Modelcited by 10
  17. 2025ReMatch: Boosting Representation through Matching for Multimodal Retrievalcited by 8
  18. 2025UniMoCo: Unified Modality Completion for Robust Multi-Modal Embeddingscited by 5
  19. 2026VidVec: Unlocking Video MLLM Embeddings for Video-Text Retrievalcited by 4
  20. 2026Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMscited by 3
  21. 2026MuCo: Multi-turn Contrastive Learning for Multimodal Embedding Modelcited by 3
  22. 2025Let Multimodal Embedders Learn When to Augment Query via Adaptive Query Augmentationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Think When Needed: Adaptive Reasoning-Driven Multimodal Embeddings with a Dual-LoRA Architecture
  2. 2025Reasoning Guided Embeddings: Leveraging MLLM Reasoning for Improved Multimodal Retrieval
  3. 2026LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck
  4. 2026Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings
  5. 2026TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrieval
  6. 2026Do Reasoning Models Enhance Embedding Models?
  7. 2026PLUME: Latent Reasoning Based Universal Multimodal Embedding
  8. 2025M2IO-R1: An Efficient RL-Enhanced Reasoning Framework for Multimodal Retrieval Augmented Multimodal Generation