mlsanityData to 14 Oct 2026

CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embedding

arXiv 2601.21262 ↗First posted 29 Jan 2026cs.CLcited by 7

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Although Multimodal Large Language Models (MLLMs) have shown remarkable potential in Visual Document Retrieval (VDR) through generating high-quality multi-vector embeddings, the substantial storage overhead caused by representing a page with thousands of visual tokens limits their practicality in real-world applications. To address this challenge, we propose an auto-regressive generation approach, CausalEmbed, for constructing multi-vector embeddings. By incorporating iterative margin loss during contrastive training, CausalEmbed encourages the embedding models to learn compact and well-structured representations. Our method enables efficient VDR tasks using only dozens of visual tokens, achieving a 30-155x reduction in token count while maintaining highly competitive performance across various backbones and benchmarks. Theoretical analysis and empirical results demonstrate the unique advantages of auto-regressive embedding generation in terms of training efficiency and scalability at test time. As a result, CausalEmbed introduces a flexible test-time scaling strategy for multi-vector VDR representations and sheds light on the generative paradigm within multimodal document retrieval. Our code is available at https://github.com/Z1zs/Causal-Embed.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 66 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

55 of 66 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding11 Nov
  2. 2025UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings1 Nov
  3. 2025RzenEmbed: Towards Comprehensive Multimodal Retrieval31 Oct
  4. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMs22 Dec
  5. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks7 Oct
  6. 2024ColPali: Efficient Document Retrieval with Vision Language Models27 Jun

Other references · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  11. 2025Gemma 3 Technical Reportcited by 1,824
  12. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
37 more
  1. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  2. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  3. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  4. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  5. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  6. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  7. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  8. 2022Matryoshka Representation Learningcited by 282
  9. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
  10. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 116
  11. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 110
  12. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 76
  13. 2024VISTA: Visualized Text Embedding For Universal Multi-Modal Retrievalcited by 67
  14. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 61
  15. 2025LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learningcited by 55
  16. 2025Think Then Embed: Generative Context Improves Multimodal Embeddingcited by 41
  17. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 39
  18. 2024MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encodingscited by 32
  19. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
  20. 2025CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuningcited by 31
  21. 2025Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Modelcited by 27
  22. 2024OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generationcited by 26
  23. 2025Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddingscited by 26
  24. 2026ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarioscited by 26
  25. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 20
  26. 2025JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generationcited by 15
  27. 2025Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrievalcited by 11
  28. 2025Let LLMs Speak Embedding Languages: Generative Text Embeddings via Iterative Contrastive Refinementcited by 10
  29. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  30. 2025Reasoning Guided Embeddings: Leveraging MLLM Reasoning for Improved Multimodal Retrievalcited by 9
  31. 2025Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrievalcited by 8
  32. 2024Starbucks-v2: Improved Training for 2D Matryoshka Embeddingscited by 7
  33. 2026ReinPool: Reinforcement Learning Pooling Multi-Vector Embeddings for Retrieval Systemcited by 5
  34. 2025Survey on Question Answering over Visually Rich Documents: Methods, Challenges, and Trendscited by 4
  35. 2025M3DR: Towards Universal Multilingual Multimodal Document Retrievalcited by 3
  36. 2025Composed Multi-modal Retrieval: A Survey of Approaches and Applicationscited by 2
  37. 2025Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Surveycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
  2. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  3. 2026Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrievalcited by 8
  4. 2026Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representationscited by 4
  5. 2026MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Frameworkcited by 2
  6. 2026PIXELRAG: Web Screenshots Beat Text for Retrieval-Augmented Generationcited by 2
  7. 2026AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrievalcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search
  2. 2026KoVRE: Training an Efficient Embedding Model for Korean Visual Document Retrieval
  3. 2026EVIE: Evidence-Vector-Informed Embeddings for Visual Document Retrieval ↗
  4. 2025Vector embedding of multi-modal texts: a tool for discovery?
  5. 2026MURE: Hierarchical Multi-Resolution Encoding via Vision-Language Models for Visual Document Retrieval
  6. 2025ABC: Achieving Better Control of Multimodal Embeddings using VLMs
  7. 2026Unveil: Unified Visual-Textual Integration and Distillation for Multi-modal Document Retrieval
  8. 2025ModernVBERT: Towards Smaller Visual Document Retrievers