mlsanityData to 7 Oct 2026

Mull-Tokens: Modality-Agnostic Latent Thinking

arXiv 2512.10941 ↗First posted 11 Dec 2025cs.CV cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

Reasoning goes beyond language; the real world requires reasoning about space, time, affordances, and much more that words alone cannot convey. Existing multimodal models exploring the potential of reasoning with images are brittle and do not scale. They rely on calling specialist tools, costly generation of images, or handcrafted reasoning data to switch between text and image thoughts. Instead, we offer a simpler alternative -- Mull-Tokens -- modality-agnostic latent tokens pre-trained to hold intermediate information in either image or text modalities to let the model think free-form towards the correct answer. We investigate best practices to train Mull-Tokens inspired by latent reasoning frameworks. We first train Mull-Tokens using supervision from interleaved text-image traces, and then fine-tune without any supervision by only using the final answers. Across four challenging spatial reasoning benchmarks involving tasks such as solving puzzles and taking different perspectives, we demonstrate that Mull-Tokens improve upon several baselines utilizing text-only reasoning or interleaved image-text reasoning, achieving a +3% average improvement and up to +16% on a puzzle solving reasoning-heavy split compared to our strongest baseline. Adding to conversations around challenges in grounding textual and visual reasoning, Mull-Tokens offers a simple solution to abstractly think in multiple modalities.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 57 of its 79 references that are in the map. First posted on arXiv 11 Dec 2025.

Builds on

57 of 79 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2025Qwen3-VL Technical Reportcited by 2,534
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  9. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  10. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  11. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  12. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
38 more
  1. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  2. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  3. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  4. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  5. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  6. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  7. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  8. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  9. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  10. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  11. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
  12. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  13. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  14. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
  15. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 204
  16. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  17. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
  18. 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 144
  19. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 120
  20. 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generationcited by 112
  21. 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worsecited by 107
  22. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  23. 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoningcited by 78
  24. 2025Thinking with Generated Imagescited by 71
  25. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Modelscited by 68
  26. 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Modelscited by 64
  27. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 58
  28. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 55
  29. 2023COLA: A Benchmark for Compositional Text-to-image Retrievalcited by 53
  30. 2025Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuningcited by 44
  31. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  32. 2025Hybrid Latent Reasoning via Reinforcement Learningcited by 30
  33. 2025SITE: towards Spatial Intelligence Thorough Evaluationcited by 26
  34. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 26
  35. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  36. 2025The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gainscited by 25
  37. 2025Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulationscited by 21
  38. 2025Lost in Embeddings: Information Loss in Vision-Language Modelscited by 18

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

  1. 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Drivingcited by 26
  2. 2026Forest Before Trees: Latent Superposition for Efficient Visual Reasoningcited by 16
  3. 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 10
  4. 2026HyLaR: Hybrid Latent Reasoning with Decoupled Policy Optimizationcited by 6
  5. 2026Semantic-Enriched Latent Visual Reasoningcited by 3
  6. 2026Ablate-to-Validate: Are Vision-Language Models Really Using Continuous Thought Tokens?cited by 1
  7. 2026DeepLatent: Think with Images via Parallel Latent Visual Reasoningcited by 1
  8. 2026GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understandingcited by 1
  9. 2026STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Modelscited by 0
  10. 2026Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Modelscited by 0
  11. 2026APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughtscited by 0
  12. 2026Perception Before Reasoning: Dynamic Latent Reasoning for Video Understanding and Question Answeringcited by 0
3 more
  1. 2026Uni-LaDiR: Latent Diffusion Unifies Multimodal Reasoningcited by 0
  2. 2026Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Spacecited by 0
  3. 2026Soft Spatial Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
  2. 2026Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence
  3. 2025Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
  4. 2026UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
  5. 2026Leveraging Latent Visual Reasoning in Silence
  6. 2026Beyond Visual Memory: Mechanistic Diagnostics of Latent Visual Reasoning
  7. 2025ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation
  8. 2026GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning