mlsanityData to 14 Oct 2026

TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrieval

arXiv 2603.02929 ↗First posted 3 Mar 2026cs.CVcited by 7

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Universal Multimodal Retrieval requires unified embedding models capable of interpreting diverse user intents, ranging from simple keywords to complex compositional instructions. While Multimodal Large Language Models (MLLMs) possess strong reasoning capabilities, prevailing adaptations confine them to static encoders, underutilizing their generative potential. This encoder-only paradigm struggles with complex intents that demand logical deduction rather than superficial pattern matching. To address this, we introduce TRACE (Task-adaptive Reasoning And Compressing Embeddings). TRACE unifies generative reasoning with discriminative representation learning. It first generates a structured Chain-of-Thought (CoT) to explicitly reason about the query, and subsequently compresses this reasoning trace into a compact embedding via a dedicated token. To train this framework, we construct M-BEIR-CoT, a large-scale dataset featuring a difficulty-aware routing strategy. Experiments on the M-BEIR benchmark establish TRACE as the new state-of-the-art. Crucially, TRACE demonstrates a learned implicit routing behavior. It autonomously activates reasoning for complex queries while bypassing it for simpler ones, achieving an optimal balance between retrieval accuracy and inference throughput. Furthermore, by internalizing the deductive process, TRACE exhibits remarkable zero-shot transferability to unseen domains and novel constraints.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 53 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

51 of 53 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval15 Dec
  2. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  3. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
  4. 2023DetGPT: Detect What You Need via Reasoning23 May

Other references · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  12. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
35 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  2. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  3. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  4. 2016Visual Dialogcited by 377
  5. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  6. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  7. 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 300
  8. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 251
  9. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
  10. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
  11. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  12. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  13. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  14. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 176
  15. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
  16. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 166
  17. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 161
  18. 2017Automatic Spatially-aware Fashion Concept Discoverycited by 134
  19. 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 126
  20. 2020Visual News: Benchmark and Challenges in News Image Captioningcited by 121
  21. 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
  22. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 113
  23. 2021WebQA: Multihop and Multimodal QAcited by 109
  24. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  25. 2024EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameterscited by 99
  26. 2021Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformerscited by 93
  27. 2024MegaPairs: Massive Data Synthesis For Universal Multimodal Retrievalcited by 84
  28. 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
  29. 2020Compositional Learning of Image-Text Query for Image Retrievalcited by 57
  30. 2023GeneCIS: A Benchmark for General Conditional Image Similaritycited by 47
  31. 2023EDIS: Entity-Driven Image Search over Multimodal Web Contentcited by 27
  32. 2024Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrievalcited by 13
  33. 2025ViSTA: Visual Storytelling using Multi-modal Adapters for Text-to-Image Diffusion Modelscited by 3
  34. 2026ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrievalcited by 3
  35. 2026WISER: Wider Search, Deeper Thinking, and Adaptive Fusion for Training-Free Zero-Shot Composed Image Retrievalcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning Guided Embeddings: Leveraging MLLM Reasoning for Improved Multimodal Retrieval
  2. 2026Think When Needed: Adaptive Reasoning-Driven Multimodal Embeddings with a Dual-LoRA Architecture
  3. 2026Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval
  4. 2026Reasoning-Augmented Representations for Multimodal Retrieval
  5. 2026Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings
  6. 2026UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval
  7. 2025Think Then Embed: Generative Context Improves Multimodal Embedding
  8. 2025Reason to Contrast: A Cascaded Multimodal Retrieval Framework