TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrieval
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Universal Multimodal Retrieval requires unified embedding models capable of interpreting diverse user intents, ranging from simple keywords to complex compositional instructions. While Multimodal Large Language Models (MLLMs) possess strong reasoning capabilities, prevailing adaptations confine them to static encoders, underutilizing their generative potential. This encoder-only paradigm struggles with complex intents that demand logical deduction rather than superficial pattern matching. To address this, we introduce TRACE (Task-adaptive Reasoning And Compressing Embeddings). TRACE unifies generative reasoning with discriminative representation learning. It first generates a structured Chain-of-Thought (CoT) to explicitly reason about the query, and subsequently compresses this reasoning trace into a compact embedding via a dedicated token. To train this framework, we construct M-BEIR-CoT, a large-scale dataset featuring a difficulty-aware routing strategy. Experiments on the M-BEIR benchmark establish TRACE as the new state-of-the-art. Crucially, TRACE demonstrates a learned implicit routing behavior. It autonomously activates reasoning for complex queries while bypassing it for simpler ones, achieving an optimal balance between retrieval accuracy and inference throughput. Furthermore, by internalizing the deductive process, TRACE exhibits remarkable zero-shot transferability to unseen domains and novel constraints.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 53 references that are in the map. First posted on arXiv 3 Mar 2026.
Builds on
51 of 53 references.
Closest ancestors · 4
Same direction, newest first
- 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval15 Dec
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
- 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
- 2023DetGPT: Detect What You Need via Reasoning23 May
Other references · 47
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Visual Instruction Tuningcited by 7,869
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
35 more
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
- 2016Visual Dialogcited by 377
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
- 2022Scaling Language-Image Pre-training via Maskingcited by 317
- 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 300
- 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 251
- 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
- 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
- 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
- 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
- 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
- 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 176
- 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
- 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 166
- 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 161
- 2017Automatic Spatially-aware Fashion Concept Discoverycited by 134
- 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 126
- 2020Visual News: Benchmark and Challenges in News Image Captioningcited by 121
- 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
- 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 113
- 2021WebQA: Multihop and Multimodal QAcited by 109
- 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
- 2024EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameterscited by 99
- 2021Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformerscited by 93
- 2024MegaPairs: Massive Data Synthesis For Universal Multimodal Retrievalcited by 84
- 2023Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewritingcited by 69
- 2020Compositional Learning of Image-Text Query for Image Retrievalcited by 57
- 2023GeneCIS: A Benchmark for General Conditional Image Similaritycited by 47
- 2023EDIS: Entity-Driven Image Search over Multimodal Web Contentcited by 27
- 2024Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrievalcited by 13
- 2025ViSTA: Visual Storytelling using Multi-modal Adapters for Text-to-Image Diffusion Modelscited by 3
- 2026ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrievalcited by 3
- 2026WISER: Wider Search, Deeper Thinking, and Adaptive Fusion for Training-Free Zero-Shot Composed Image Retrievalcited by 3
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026PLUME: Latent Reasoning Based Universal Multimodal Embeddingcited by 11
- 2026MOON3.0: Reasoning-aware Multimodal Representation Learning for E-commerce Product Understandingcited by 4
- 2026Douyin Multimodal Embedding Model Technical Reportcited by 2
- 2026ReLoop-UME: Recurrent Depth with Learnable Retrieval Registers for Universal Multimodal Embeddingcited by 0
- 2026PailitaoGR: Latent Think-with-Images for Generative Image Retrievalcited by 0
- 2026Is Reasoning Always Useful? Rethinking Reasoning Utility in Universal Multimodal Embeddingscited by 0
- 2026Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrievalcited by 0
- 2026Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval8 Oct 2026
- 2026PailitaoGR: Latent Think-with-Images for Generative Image Retrieval27 Aug 2026
- 2026Is Reasoning Always Useful? Rethinking Reasoning Utility in Universal Multimodal Embeddings26 Aug 2026
- 2026Douyin Multimodal Embedding Model Technical Report3 Aug 2026
- 2026ReLoop-UME: Recurrent Depth with Learnable Retrieval Registers for Universal Multimodal Embedding30 Jul 2026
- 2026PLUME: Latent Reasoning Based Universal Multimodal Embedding2 Apr 2026
- 2026MOON3.0: Reasoning-aware Multimodal Representation Learning for E-commerce Product Understanding1 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025Reasoning Guided Embeddings: Leveraging MLLM Reasoning for Improved Multimodal Retrieval
- 2026Think When Needed: Adaptive Reasoning-Driven Multimodal Embeddings with a Dual-LoRA Architecture
- 2026Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval
- 2026Reasoning-Augmented Representations for Multimodal Retrieval
- 2026Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings
- 2026UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval
- 2025Think Then Embed: Generative Context Improves Multimodal Embedding
- 2025Reason to Contrast: A Cascaded Multimodal Retrieval Framework