ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Multimodal Large Language Models (MLLMs) have achieved remarkable progress but still struggle with complex visual reasoning tasks requiring multi-step perception and logical deduction. While explicit visual generation incurs prohibitive computational costs, existing latent approaches often rely on external experts or lack rigorous cognitive logic. In this paper, we introduce ProLaViT (Progressive Latent Visual Thought), a framework empowering MLLMs to perform structured visual derivation in the continuous latent space. Unlike works dependent on heterogeneous external models, ProLaViT leverages an endogenous self-distillation mechanism, utilizing the model's own visual encoder to supervise latent thoughts. To facilitate this, we construct a scalable programmatic synthesis pipeline enabling the model to internalize algorithmic precision without inference time tools. We design two reasoning paradigms: (1) Coarse-to-Fine Causal Chain for spatial tasks, guiding attention from global context to local targets. (2) Dialectical Reasoning Chain for logical tasks, incorporating counter-factual thinking for verification. Furthermore, we propose a Distance-Weighted Diversity Loss to impose topology-aware constraints, preventing feature degeneration by enforcing semantic distinctiveness. Extensive experiments demonstrate that ProLaViT outperforms baselines on vision-centric benchmarks, achieving superior accuracy and interpretability with high efficiency.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 37 of its 39 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
37 of 39 references.
Closest ancestors · 16
Same direction, newest first
- 2025Interleaved Latent Visual Reasoning with Selective Perceptual Modeling5 Dec
- 2025Monet: Reasoning in Latent Visual Space Beyond Images and Language26 Nov
- 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens24 Nov
- 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning30 Oct
- 2025Latent Visual Reasoning29 Sep
- 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning22 Jul
- 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens20 Jun
- 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency13 Feb
- 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action20 Mar
4 more
- 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models8 Mar
- 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar
Other references · 21
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Segment Anythingcited by 8,234
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
9 more
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
- 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
- 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
- 2023Generating Images with Multimodal Language Modelscited by 260
- 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
- 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 185
- 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generationcited by 114
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
- 2026DeepLatent: Think with Images via Parallel Latent Visual Reasoning
- 2026CrystaL: Spontaneous Emergence of Visual Latents in MLLMs
- 2026Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs
- 2026SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows
- 2026LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning
- 2026UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
- 2026Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language Model