mlsanityData to 7 Oct 2026

Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

arXiv 2608.15869 ↗First posted 16 Aug 2026cs.CV cs.AI cs.CLcited by 1

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments. By generating intermediate reasoning images, Visual CoT provides an intuitive mechanism for visual foresight but introduces substantial inference overhead, which is particularly problematic for proactive video reasoning. We ask whether models can learn to think visually during training while reasoning directly at inference. We introduce Internalized Visual Thinking (IVT), a post-training framework that jointly optimizes textual prediction and next-embedding prediction over unlabeled videos. Given a partially observed video, IVT predicts latent representations of future frames together with the target textual answer, encouraging the model to capture motion, object transitions, interactions, and latent intent. At inference, IVT generates the answer directly without synthesizing or re-encoding future frames. We conduct controlled studies across target representations, decoder designs, prediction horizons, data mixtures, training curricula, and predictive objectives. IVT improves over direct-answer fine-tuning on all six evaluation settings while retaining the same inference pathway. Compared with explicit Visual CoT, IVT achieves comparable or better performance and reduces average end-to-end latency by more than 5x. Together, our findings suggest that explicit pixel-space generation at inference time, as used in visual chain-of-thought, may not be necessary for effective proactive video reasoning. Predictive world modeling can be internalized during training to produce multimodal reasoners that are both more accurate and substantially more efficient.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 56 of its 61 references that are in the map. First posted on arXiv 16 Aug 2026.

Builds on

56 of 61 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs13 Feb
  2. 2026AbductiveMLLM: Boosting Visual Abductive Reasoning Within MLLMs6 Jan
  3. 2025Video-as-Answer: Predict and Generate Next Video Event with Joint-GRPO20 Nov
  4. 2025Latent Visual Reasoning29 Sep
  5. 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning22 Jul
  6. 2025Thinking with Generated Images28 May
  7. 2025Fostering Video Reasoning via Next-Event Prediction28 May
  8. 2025Grounded Chain-of-Thought for Multimodal Large Language Models17 Mar
  9. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  10. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
  11. 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models22 May
  12. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May
3 more
  1. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
  2. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
  3. 2022Visual Abductive Reasoning26 Mar

Other references · 41

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  5. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  6. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  7. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  8. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  9. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  10. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  11. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  12. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
29 more
  1. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  2. 2020Rescaling Egocentric Visioncited by 529
  3. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  4. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  5. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  6. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  7. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  8. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  9. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 290
  10. 2021Anticipative Video Transformercited by 197
  11. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
  12. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 119
  13. 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
  14. 2024Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Modelscited by 65
  15. 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 61
  16. 2022Future Transformer for Long-term Action Anticipationcited by 57
  17. 2017Encouraging LSTMs to Anticipate Actions Very Earlycited by 37
  18. 2024Can't make an Omelette without Breaking some Eggs: Plausible Action Anticipation using Large Video-Language Modelscited by 32
  19. 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
  20. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 32
  21. 2025ProactiveVideoQA: A Comprehensive Benchmark Evaluating Proactive Interactions in Video Large Language Modelscited by 24
  22. 2025Next-Embedding Prediction Makes Strong Vision Learnerscited by 21
  23. 2022The Wisdom of Crowds: Temporal Progressive Attention for Early Action Predictioncited by 13
  24. 2024Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Modelscited by 12
  25. 2025UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in Reinforcement Learningcited by 8
  26. 2023STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action Recognitioncited by 7
  27. 2026UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?cited by 6
  28. 2026Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMscited by 6
  29. 2021Review of Video Predictive Understanding: Early Action Recognition and Future Action Predictioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models
  2. 2026Act2See: Emergent Active Visual Perception for Video Reasoning
  3. 2025Rethinking Chain-of-Thought Reasoning for Videos
  4. 2025When Thinking Drifts: Evidential Grounding for Robust Video Reasoning
  5. 2026VTI-CoT: Visual-Textual Interleaved Chain of Thought for Video Reasoning
  6. 2026VideoLatent: Video-Language Learning via Latent Self-Forcing
  7. 2025Monet: Reasoning in Latent Visual Space Beyond Images and Language
  8. 2025Video Finetuning Improves Reasoning Between Frames