mlsanityData to 7 Oct 2026

TSEmbed: Unlocking Task Scaling in Universal Multimodal Embeddings

arXiv 2603.04772 ↗First posted 5 Mar 2026cs.CL cs.AIcited by 6

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Despite the exceptional reasoning capabilities of Multimodal Large Language Models (MLLMs), their adaptation into universal embedding models is significantly impeded by task conflict. To address this, we propose TSEmbed, a universal multimodal embedding framework that synergizes Mixture-of-Experts (MoE) with Low-Rank Adaptation (LoRA) to explicitly disentangle conflicting task objectives. Moreover, we introduce Expert-Aware Negative Sampling (EANS), a novel strategy that leverages expert routing distributions as an intrinsic proxy for semantic similarity. By dynamically prioritizing informative hard negatives that share expert activation patterns with the query, EANS effectively sharpens the model's discriminative power and refines embedding boundaries. To ensure training stability, we further devise a two-stage learning paradigm that solidifies expert specialization before optimizing representations via EANS. TSEmbed achieves state-of-the-art performance on both the Massive Multimodal Embedding Benchmark (MMEB) and real-world industrial production datasets, laying a foundation for task-level scaling in universal multimodal embeddings.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 50 of its 54 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

50 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddings14 Feb
  2. 2025Bridging the Copyright Gap: Do Large Vision-Language Models Recognize and Respect Copyrighted Content?26 Dec
  3. 2025Think Then Embed: Generative Context Improves Multimodal Embedding6 Oct

Other references · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  9. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  10. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  11. 2017A Survey on Multi-Task Learningcited by 838
  12. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
35 more
  1. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  2. 2020Contrastive Learning with Hard Negative Samplescited by 619
  3. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  4. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  5. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  6. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  7. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  8. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  9. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 190
  10. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
  11. 2019AdaShare: Learning What To Share For Efficient Deep Multi-Task Learningcited by 159
  12. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  13. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 123
  14. 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 123
  15. 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
  16. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 110
  17. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  18. 2021Multi-Task Learning in Natural Language Processing: An Overviewcited by 61
  19. 2022Image-text Retrieval: A Survey on Recent Research and Developmentcited by 55
  20. 2025LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learningcited by 54
  21. 2022Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCocited by 52
  22. 2024NoteLLM-2: Multimodal Large Representation Models for Recommendationcited by 44
  23. 2025UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learningcited by 41
  24. 2024Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendationcited by 33
  25. 2025CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuningcited by 31
  26. 2025RzenEmbed: Towards Comprehensive Multimodal Retrievalcited by 28
  27. 2025Breaking the Batch Barrier (B3) of Contrastive Learning via Smart Batch Miningcited by 27
  28. 2025Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrievalcited by 26
  29. 2024Multimodal Pretraining, Adaptation, and Generation for Recommendation: A Surveycited by 24
  30. 2025Improve Multi-Modal Embedding Learning via Explicit Hard Negative Gradient Amplifyingcited by 21
  31. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  32. 2024CopyrightMeter: Revisiting Copyright Protection in Text-to-image Modelscited by 6
  33. 2025Memory-Efficient Federated Fine-Tuning of Large Language Models via Layer Pruningcited by 5
  34. 2025Elastic Mixture of Rank-Wise Experts for Knowledge Reuse in Federated Fine-Tuningcited by 5
  35. 2025Learning Like Humans: Resource-Efficient Federated Fine-Tuning through Cognitive Developmental Stagescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddingscited by 5
  2. 2026"I See What You Did There": Can Large Vision-Language Models Understand Multimodal Puns?cited by 1
  3. 2026Spotlight and Shadow: Attention-Guided Dual-Anchor Introspective Decoding for MLLM Hallucination Mitigationcited by 1
  4. 2026MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Modelscited by 0
  5. 2026VaME: Exploring Variational Latent Reasoning for Multimodal Embeddingscited by 0
  6. 2026CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Generator to Embedder: Harnessing Innate Abilities of Multimodal LLMs via Building Zero-Shot Discriminative Embedding Model
  2. 2026MMEmb-R1: Reasoning-Enhanced Multimodal Embedding with Pair-Aware Selection and Adaptive Control
  3. 2026Think When Needed: Adaptive Reasoning-Driven Multimodal Embeddings with a Dual-LoRA Architecture
  4. 2025Resolving Task Objective Conflicts in Unified Model via Task-Aware Mixture-of-Experts
  5. 2025Reasoning Guided Embeddings: Leveraging MLLM Reasoning for Improved Multimodal Retrieval
  6. 2025UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings
  7. 2026Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
  8. 2026Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation