Spatial-ViLT: Enhancing Visual Spatial Reasoning through Multi-Task Learning
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Vision-language models (VLMs) have advanced multimodal reasoning but still face challenges in spatial reasoning for 3D scenes and complex object configurations. To address this, we introduce SpatialViLT, an enhanced VLM that integrates spatial features like depth maps, 3D coordinates, and edge maps through a multi-task learning framework. This approach enriches multimodal embeddings with spatial understanding. We propose two variants: SpatialViLT and MaskedSpatialViLT, focusing on full and masked object regions, respectively. Additionally, SpatialEnsemble combines both approaches, achieving state-of-the-art accuracy. Our models excel in spatial reasoning categories such as directional, topological, and proximity relations, as demonstrated on the challenging Visual Spatial Reasoning (VSR) dataset. This work represents a significant step in enhancing the spatial intelligence of AI systems, crucial for advanced multimodal understanding and real-world applications.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 27 references that are in the map. First posted on arXiv 3 Oct 2025.
Builds on
22 of 27 references.
Closest ancestors · 7
Same direction, newest first
- 2024TopViewRS: Vision-Language Models as Top-View Spatial Reasoners4 Jun
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
- 2024Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning23 May
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
- 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct
- 2023Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Models18 Aug
- 2022Visual Spatial Reasoning30 Apr
Other references · 15
- 2017Mask R-CNNcited by 7,259
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
- 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
- 2018Open3D: A Modern Library for 3D Data Processingcited by 717
- 2021Image Segmentation Using Text and Image Promptscited by 509
- 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
- 2020Monocular Human Pose Estimation: A Survey of Deep Learning-based Methodscited by 59
- 2023Towards In-context Scene Understandingcited by 45
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026LAST: Leveraging Tools as Hints to Enhance Spatial Reasoning for Multimodal Large Language Models
- 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
- 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
- 2025SpatialMosaic: A Multiview VLM Dataset for Partial Visibility
- 2026INTCORT: Training-Free Spatial Reasoning Enhancement for Vision-Language Models via Input Transformations and Confidence Routing
- 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
- 2024I Know About "Up"! Enhancing Spatial Reasoning in Visual Language Models Through 3D Reconstruction
- 2025SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models