# Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens

arXiv: https://arxiv.org/abs/2511.19418 · HTML: https://mlsanity.com/paper/2511.19418/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 38 of its 73 references that are in the map.
- First posted on arXiv 24 Nov 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 38)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?](https://mlsanity.com/paper/2408.13257/) (2024)
- [We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?](https://mlsanity.com/paper/2407.01284/) (2024)
- [Think before you speak: Training Language Models With Pause Tokens](https://mlsanity.com/paper/2310.02226/) (2023)
- [Chain-of-Thought Reasoning Without Prompting](https://mlsanity.com/paper/2402.10200/) (2024)
- [Imagine while Reasoning in Space: Multimodal Visualization-of-Thought](https://mlsanity.com/paper/2501.07542/) (2025)
- [Compressed Chain of Thought: Efficient Reasoning Through Dense Representations](https://mlsanity.com/paper/2412.13171/) (2024)
- [MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI](https://mlsanity.com/paper/2404.16006/) (2024)
- [MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding](https://mlsanity.com/paper/2406.09411/) (2024)
- [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://mlsanity.com/paper/2311.01460/) (2023)
- [Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models](https://mlsanity.com/paper/2408.15556/) (2024)
- [Vision language models are blind: Failing to translate detailed visual features into words](https://mlsanity.com/paper/2407.06581/) (2024)
- [Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens](https://mlsanity.com/paper/2506.17218/) (2025)
- [Latent Visual Reasoning](https://mlsanity.com/paper/2509.24251/) (2025)

## Built on by (most cited first, 30 of 71)

- [The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook](https://mlsanity.com/paper/2604.02029/) (2026)
- [DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models](https://mlsanity.com/paper/2512.24165/) (2025)
- [Vision-aligned Latent Reasoning for Multi-modal Large Language Model](https://mlsanity.com/paper/2602.04476/) (2026)
- [LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning](https://mlsanity.com/paper/2601.10129/) (2026)
- [On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs](https://mlsanity.com/paper/2602.12506/) (2026)
- [Dual Latent Memory for Visual Multi-agent System](https://mlsanity.com/paper/2602.00471/) (2026)
- [Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomalies](https://mlsanity.com/paper/2602.01816/) (2026)
- [Show, Don't Tell: Morphing Latent Reasoning into Image Generation](https://mlsanity.com/paper/2602.02227/) (2026)
- [UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs](https://mlsanity.com/paper/2605.11856/) (2026)
- [LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning](https://mlsanity.com/paper/2605.22012/) (2026)
- [S2GR: Stepwise Semantic-Guided Reasoning in Latent Space for Generative Recommendation](https://mlsanity.com/paper/2601.18664/) (2026)
- [CrystaL: Spontaneous Emergence of Visual Latents in MLLMs](https://mlsanity.com/paper/2602.20980/) (2026)
- [EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models](https://mlsanity.com/paper/2603.12252/) (2026)
- [Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs](https://mlsanity.com/paper/2605.02735/) (2026)
- [Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning](https://mlsanity.com/paper/2605.07106/) (2026)
- [Leveraging Latent Visual Reasoning in Silence](https://mlsanity.com/paper/2605.18641/) (2026)
- [Beyond Visual Memory: Mechanistic Diagnostics of Latent Visual Reasoning](https://mlsanity.com/paper/2606.01287/) (2026)
- [SwimBird: Eliciting Switchable Reasoning Mode in Hybrid Autoregressive MLLMs](https://mlsanity.com/paper/2602.06040/) (2026)
- [What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis](https://mlsanity.com/paper/2602.12395/) (2026)
- [Multimodal Latent Reasoning via Predictive Embeddings](https://mlsanity.com/paper/2604.08065/) (2026)
- [Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2605.12374/) (2026)
- [Foveated Reasoning: Stateful, Action-based Visual Focusing for Vision-Language Models](https://mlsanity.com/paper/2604.21079/) (2026)
- [ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both](https://mlsanity.com/paper/2605.15198/) (2026)
- [Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction](https://mlsanity.com/paper/2606.05769/) (2026)
- [ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space](https://mlsanity.com/paper/2607.02907/) (2026)
- [Evidence-RL: Towards Evidence-intensive Visual Reasoning](https://mlsanity.com/paper/2608.08021/) (2026)
- [PhysMem: Scaling Test-Time Memory for Embodied Physical Reasoning](https://mlsanity.com/paper/2602.20323/) (2026)
- [Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMs](https://mlsanity.com/paper/2603.06697/) (2026)
- [Agentic AI for Remote Sensing: Technical Challenges and Research Directions](https://mlsanity.com/paper/2604.24919/) (2026)
- [Audio-Visual Intelligence in Large Foundation Models](https://mlsanity.com/paper/2605.04045/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) excel at reasoning in linguistic space but struggle with perceptual understanding that requires dense visual perception, e.g., spatial reasoning and geometric awareness. This limitation stems from the fact that current VLMs have limited mechanisms to capture dense visual information across spatial dimensions. We introduce Chain-of-Visual-Thought (COVT), a framework that enables VLMs to reason not only in words but also through continuous visual tokens-compact latent representations that encode rich perceptual cues. Within a small budget of roughly 20 tokens, COVT distills knowledge from lightweight vision experts, capturing complementary properties such as 2D appearance, 3D geometry, spatial layout, and edge structure. During training, the VLM with COVT autoregressively predicts these visual tokens to reconstruct dense supervision signals (e.g., depth, segmentation, edges, and DINO features). At inference, the model reasons directly in the continuous visual token space, preserving efficiency while optionally decoding dense predictions for interpretability. Evaluated across more than ten diverse perception benchmarks, including CV-Bench, MMVP, RealWorldQA, MMStar, WorldMedQA, and HRBench, integrating COVT into strong VLMs such as Qwen2.5-VL and LLaVA consistently improves performance by 3% to 16% and demonstrates that compact continuous visual thinking enables more precise, grounded, and interpretable multimodal intelligence.