# Sketch-in-Latents: Eliciting Unified Reasoning in MLLMs

arXiv: https://arxiv.org/abs/2512.16584 · HTML: https://mlsanity.com/paper/2512.16584/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 34 of its 54 references that are in the map.
- First posted on arXiv 18 Dec 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 34)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning](https://mlsanity.com/paper/2505.14362/) (2025)
- [VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning](https://mlsanity.com/paper/2504.08837/) (2025)
- [MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?](https://mlsanity.com/paper/2408.13257/) (2024)
- [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://mlsanity.com/paper/2502.21074/) (2025)
- [Imagine while Reasoning in Space: Multimodal Visualization-of-Thought](https://mlsanity.com/paper/2501.07542/) (2025)
- [Compressed Chain of Thought: Efficient Reasoning Through Dense Representations](https://mlsanity.com/paper/2412.13171/) (2024)
- [Thyme: Think Beyond Images](https://mlsanity.com/paper/2508.11630/) (2025)
- [WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs](https://mlsanity.com/paper/2502.04326/) (2025)
- [Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens](https://mlsanity.com/paper/2506.17218/) (2025)
- [Perception-R1: Pioneering Perception Policy with Reinforcement Learning](https://mlsanity.com/paper/2504.07954/) (2025)
- [Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space](https://mlsanity.com/paper/2505.15778/) (2025)
- [Latent Visual Reasoning](https://mlsanity.com/paper/2509.24251/) (2025)
- [Visual Planning: Let's Think Only with Images](https://mlsanity.com/paper/2505.11409/) (2025)
- [Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning](https://mlsanity.com/paper/2507.16746/) (2025)
- [PyVision: Agentic Vision with Dynamic Tooling](https://mlsanity.com/paper/2507.07998/) (2025)

## Built on by (most cited first, 25 of 25)

- [The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook](https://mlsanity.com/paper/2604.02029/) (2026)
- [Forest Before Trees: Latent Superposition for Efficient Visual Reasoning](https://mlsanity.com/paper/2601.06803/) (2026)
- [LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning](https://mlsanity.com/paper/2601.10129/) (2026)
- [Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models](https://mlsanity.com/paper/2604.08545/) (2026)
- [Imagination Helps Visual Reasoning, But Not Yet in Latent Space](https://mlsanity.com/paper/2602.22766/) (2026)
- [Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoning](https://mlsanity.com/paper/2601.21037/) (2026)
- [UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs](https://mlsanity.com/paper/2605.11856/) (2026)
- [CrystaL: Spontaneous Emergence of Visual Latents in MLLMs](https://mlsanity.com/paper/2602.20980/) (2026)
- [HyLaR: Hybrid Latent Reasoning with Decoupled Policy Optimization](https://mlsanity.com/paper/2604.20328/) (2026)
- [Leveraging Latent Visual Reasoning in Silence](https://mlsanity.com/paper/2605.18641/) (2026)
- [SwimBird: Eliciting Switchable Reasoning Mode in Hybrid Autoregressive MLLMs](https://mlsanity.com/paper/2602.06040/) (2026)
- [Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction](https://mlsanity.com/paper/2606.05769/) (2026)
- [Unlocking the Latent Canvas: Eliciting and Benchmarking Symbolic Visual Expression in LLMs](https://mlsanity.com/paper/2603.14505/) (2026)
- [ReGuLaR: Relation-Grounded Latent Reasoning for Large Vision-Language Models](https://mlsanity.com/paper/2605.30587/) (2026)
- [LUT: Latent Utility Training for Visual Reasoning](https://mlsanity.com/paper/2608.00743/) (2026)
- [Scaffolding Minds: Optimizing Latent Visual Target Representations for Multimodal Reasoning](https://mlsanity.com/paper/2608.19669/) (2026)
- [Reason Through the Latent! Making Latent Visual Reasoning Necessary](https://mlsanity.com/paper/2609.06746/) (2026)
- [GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning](https://mlsanity.com/paper/2608.15517/) (2026)
- [Assessing Adversarial Robustness of Latent Reasoning Models](https://mlsanity.com/paper/2609.22228/) (2026)
- [VideoLatent: Video-Language Learning via Latent Self-Forcing](https://mlsanity.com/paper/2606.22870/) (2026)
- [Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning](https://mlsanity.com/paper/2607.02963/) (2026)
- [OPLD: On-Policy Latent Distillation for Multimodal Reasoning](https://mlsanity.com/paper/2607.28154/) (2026)
- [MM-OPD: Towards One More Bottleneck Between Perception and Reasoning](https://mlsanity.com/paper/2609.32690/) (2026)
- [Distilling Visual Reasoning into Text Space](https://mlsanity.com/paper/2609.34408/) (2026)
- [Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence](https://mlsanity.com/paper/2609.34563/) (2026)

## Abstract (arXiv)

While Multimodal Large Language Models (MLLMs) excel at visual understanding tasks through text reasoning, they often fall short in scenarios requiring visual imagination. Unlike current works that take predefined external toolkits or generate images during thinking, however, humans can form flexible visual-text imagination and interactions during thinking without predefined toolkits, where one important reason is that humans construct the visual-text thinking process in a unified space inside the brain. Inspired by this capability, given that current MLLMs already encode visual and text information in the same feature space, we hold that visual tokens can be seamlessly inserted into the reasoning process carried by text tokens, where ideally, all visual imagination processes can be encoded by the latent features. To achieve this goal, we propose Sketch-in-Latents (SkiLa), a novel paradigm for unified multi-modal reasoning that expands the auto-regressive capabilities of MLLMs to natively generate continuous visual embeddings, termed latent sketch tokens, as visual thoughts. During multi-step reasoning, the model dynamically alternates between textual thinking mode for generating textual think tokens and visual sketching mode for generating latent sketch tokens. A latent visual semantics reconstruction mechanism is proposed to ensure these latent sketch tokens are semantically grounded. Extensive experiments demonstrate that SkiLa achieves superior performance on vision-centric tasks while exhibiting strong generalization to diverse general multi-modal benchmarks. Codes will be released at https://github.com/TungChintao/SkiLa.