# LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck

arXiv: https://arxiv.org/abs/2606.13061 · HTML: https://mlsanity.com/paper/2606.13061/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 40 of its 52 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [EVA-CLIP: Improved Training Techniques for CLIP at Scale](https://mlsanity.com/paper/2303.15389/) (2023)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Think before you speak: Training Language Models With Pause Tokens](https://mlsanity.com/paper/2310.02226/) (2023)
- [TokenSkip: Controllable Chain-of-Thought Compression in LLMs](https://mlsanity.com/paper/2502.12067/) (2025)
- [VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents](https://mlsanity.com/paper/2410.10594/) (2024)
- [VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks](https://mlsanity.com/paper/2410.05160/) (2024)
- [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://mlsanity.com/paper/2502.21074/) (2025)
- [GME: Improving Universal Multimodal Retrieval by Multimodal LLMs](https://mlsanity.com/paper/2412.16855/) (2024)
- [CoT-Valve: Length-Compressible Chain-of-Thought Tuning](https://mlsanity.com/paper/2502.09601/) (2025)
- [MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs](https://mlsanity.com/paper/2411.02571/) (2024)
- [Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward](https://mlsanity.com/paper/2404.01258/) (2024)
- [SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs](https://mlsanity.com/paper/2502.12134/) (2025)
- [VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documents](https://mlsanity.com/paper/2507.04590/) (2025)
- [LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant](https://mlsanity.com/paper/2412.01720/) (2024)
- [LightThinker: Thinking Step-by-Step Compression](https://mlsanity.com/paper/2502.15589/) (2025)
- [Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains](https://mlsanity.com/paper/2505.16552/) (2025)
- [Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs](https://mlsanity.com/paper/2504.17432/) (2025)
- [SIM-CoT: Supervised Implicit Chain-of-Thought](https://mlsanity.com/paper/2509.20317/) (2025)
- [UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning](https://mlsanity.com/paper/2510.13515/) (2025)
- [Think Then Embed: Generative Context Improves Multimodal Embedding](https://mlsanity.com/paper/2510.05014/) (2025)
- [MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction](https://mlsanity.com/paper/2509.18095/) (2025)
- [UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings](https://mlsanity.com/paper/2511.00405/) (2025)
- [RzenEmbed: Towards Comprehensive Multimodal Retrieval](https://mlsanity.com/paper/2510.27350/) (2025)

## Built on by (most cited first, 5 of 5)

- [UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval](https://mlsanity.com/paper/2608.18504/) (2026)
- [Learning Multimodal Embeddings with Evidence-Aligned Readout](https://mlsanity.com/paper/2609.33659/) (2026)
- [Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings](https://mlsanity.com/paper/2609.15296/) (2026)
- [VaME: Exploring Variational Latent Reasoning for Multimodal Embeddings](https://mlsanity.com/paper/2609.33402/) (2026)
- [Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval](https://mlsanity.com/paper/2610.11666/) (2026)

## Abstract (arXiv)

Reasoning-driven universal multimodal embedding has advanced rapidly by introducing Chain-of-Thought (CoT) reasoning into the embedding pipeline. Despite the strong performance across both general and complex tasks, this paradigm suffers from two core limitations: (i) autoregressive CoT reasoning incurs high computational cost, making it impractical for low-latency retrieval; and (ii) embedding performance is heavily coupled with CoT annotation quality, making large-scale training unreliable. These raise fundamental questions: Is textual CoT the optimal form of reasoning for embedding, and can effective embedding reasoning be accomplished in latent space? To this end, we propose LaME (Latent Reasoning Multimodal Embedding), which formulates embedding-oriented latent reasoning as a weakly supervised information bottleneck. LaME employs K learnable reason tokens as a fixed-capacity bottleneck, completing all reasoning within a single forward pass. The two weak supervision signals structurally decouple contrastive from autoregressive objectives and eliminate dependence on CoT annotations, while a two-stage training pipeline ensures stable convergence. Experiments on MMEB-v2 and MRMR show that LaME achieves competitive performance, surpassing some explicit CoT-based models, while delivering 60x faster inference than explicit CoT methods and 2x faster than latent baselines with throughput comparable to discriminative embedding models. Code is available at https://github.com/PeppaWu/LaME.