# Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillation arXiv: https://arxiv.org/abs/2606.06076 ยท HTML: https://mlsanity.com/paper/2606.06076/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 25 of its 44 references that are in the map. - First posted on arXiv 4 Jun 2026. ## Placed in - Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/) - Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/) ## Builds on (most cited first, 25 of 25) - [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025) - [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024) - [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023) - [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026) - [Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2403.16999/) (2024) - [Imagine while Reasoning in Space: Multimodal Visualization-of-Thought](https://mlsanity.com/paper/2501.07542/) (2025) - [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025) - [Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes](https://mlsanity.com/paper/2603.25562/) (2026) - [OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles](https://mlsanity.com/paper/2503.17352/) (2025) - [Visual Planning: Let's Think Only with Images](https://mlsanity.com/paper/2505.11409/) (2025) - [Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation](https://mlsanity.com/paper/2605.18740/) (2026) - [Thinking with Generated Images](https://mlsanity.com/paper/2505.22525/) (2025) - [MIRAGE: The Illusion of Visual Understanding](https://mlsanity.com/paper/2603.21687/) (2026) - [VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs](https://mlsanity.com/paper/2407.01863/) (2024) - [AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO](https://mlsanity.com/paper/2502.14669/) (2025) - [A Configurable Library for Generating and Manipulating Maze Datasets](https://mlsanity.com/paper/2309.10498/) (2023) - [VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation](https://mlsanity.com/paper/2510.23497/) (2025) - [Mini-BEHAVIOR: A Procedurally Generated Benchmark for Long-horizon Decision-Making in Embodied AI](https://mlsanity.com/paper/2310.01824/) (2023) - [KVPO: ODE-Native GRPO for Autoregressive Video Alignment via KV Semantic Exploration](https://mlsanity.com/paper/2605.14278/) (2026) - [LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model](https://mlsanity.com/paper/2604.02097/) (2026) - [When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation](https://mlsanity.com/paper/2606.03532/) (2026) - [Planning with Vision-Language Models and a Use Case in Robot-Assisted Teaching](https://mlsanity.com/paper/2501.17665/) (2025) - [ViPlan: A Benchmark for Visual Planning with Symbolic Predicates and Vision-Language Models](https://mlsanity.com/paper/2505.13180/) (2025) - [Simulation to Rules: A Dual-VLM Framework for Formal Visual Planning](https://mlsanity.com/paper/2510.03182/) (2025) - [Improving Vision-language Models with Perception-centric Process Reward Models](https://mlsanity.com/paper/2604.24583/) (2026) ## Built on by (most cited first, 2 of 2) - [RADC: Risk-Aware Dual Caching for Vision-Language Test-Time Adaptation](https://mlsanity.com/paper/2610.06932/) (2026) - [ContourVLA: A Closed-Loop Perception-Action Contour Policy for Generalized Referring Expression Segmentation](https://mlsanity.com/paper/2610.12107/) (2026) ## Abstract (arXiv) While Vision-Language Models excel at general multimodal understanding, they still struggle with visual spatial planning. We attribute this limitation to a perception--reasoning modality gap. Visual planning requires models to infer latent state structures from pixels and then reason over the recovered structure to produce valid actions, whereas symbolic planning directly leverages explicit representation. This discrepancy introduces two sequential bottlenecks: visual state recovery at the perception stage and multi-step planning at the reasoning stage. To address this, we propose MGSD, a two-stage modality-gap-aware self-distillation framework. First, a cold-start grounding stage establishes reliable visual state recovery before on-policy training. Second, a symbol-guided on-policy self-distillation stage transfers the privileged teacher's planning behavior to the student through token-level supervision on student-generated prefixes. Crucially, symbolic information is used only during training, while inference relies exclusively on visual inputs. Experiments on visual planning benchmarks show that MGSD consistently improves performance across different model scales, raising the macro average by 19.3% and 18.4%, respectively. The resulting models substantially reduce the gap to the upper bounds obtained with symbolic inputs. Ablation studies and diagnostic analyses further confirm that the gains arise from improvements in both visual state recovery and optimal-path reasoning. These results demonstrate that MGSD strengthens not only the recovery of actionable states from visual observations but also the ability to plan over the inferred structures. Code is available at https://github.com/Oranger-l/MGSD.