mlsanityData to 14 Oct 2026

Distilling 3D Spatial Reasoning into a Lightweight Vision-Language Model with CoT

arXiv 2605.09719 ↗First posted 10 May 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

also in 2 more
Abstract (arXiv)

Large-scale 3D vision-language models (VLMs) like LLaVA-3D offer strong spatial reasoning but are difficult to deploy due to high computational costs. We propose a knowledge distillation framework that transfers spatial reasoning from a 7B teacher to a 2.29B student model. Our approach achieves 8.7x lower inference latency and a 3x reduction in model size while retaining 54-72% of the teacher's performance. The framework utilizes VGGT as the vision encoder and a multi-task distillation pipeline with uncertainty-aware loss weighting. To improve reasoning without chain-of-thought (CoT) data, we introduce "Hidden CoT": learnable latent tokens that serve as an internal scratchpad before answer generation. This is the first use of latent scratchpad reasoning in distilled 3D VLMs. The student model jointly performs spatial description, depth estimation, and object detection. Experiments on ScanNet and 3D-FRONT show strong spatial understanding, reaching 68-72% accuracy on proximity and contact tasks. Our framework enables efficient 3D scene QA on resource-constrained platforms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 35 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

31 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark10 Dec

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  11. 2015VQA: Visual Question Answeringcited by 2,506
  12. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
18 more
  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  2. 2020Knowledge Distillation: A Surveycited by 1,603
  3. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  4. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  5. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  6. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  7. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  8. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  9. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  10. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 122
  11. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 117
  12. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 110
  13. 2021Learning Efficient Vision Transformers via Fine-Grained Manifold Distillationcited by 60
  14. 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognitioncited by 35
  15. 2024Distilling Vision-Language Models on Millions of Videoscited by 20
  16. 2023Knowledge Distillation in Vision Transformers: A Critical Reviewcited by 13
  17. 2023MOFA: A Model Simplification Roadmap for Image Restoration on Mobile Devicescited by 5
  18. 2025PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detectorcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 20253D-Aware Vision-Language Models Fine-Tuning with Geometric Distillation
  2. 2026Multi-View Relational Distillation for Spatial Reasoning with Vision-Language Models
  3. 2026Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
  4. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors
  5. 2025N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Models
  6. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D
  7. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
  8. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities