mlsanityData to 14 Oct 2026

I Know About "Up"! Enhancing Spatial Reasoning in Visual Language Models Through 3D Reconstruction

arXiv 2407.14133 ↗First posted 19 Jul 2024cs.CLcited by 4

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Visual Language Models (VLMs) are essential for various tasks, particularly visual reasoning tasks, due to their robust multi-modal information integration, visual reasoning capabilities, and contextual awareness. However, existing VLMs' visual spatial reasoning capabilities are often inadequate, struggling even with basic tasks such as distinguishing left from right. To address this, we propose the ours model, designed to enhance the visual spatial reasoning abilities of VLMS. ZeroVLM employs Zero-1-to-3, a 3D reconstruction model for obtaining different views of the input images and incorporates a prompting mechanism to further improve visual spatial reasoning. Experimental results on four visual spatial reasoning datasets show that our ours achieves up to 19.48% accuracy improvement, which indicates the effectiveness of the 3D reconstruction and prompting mechanisms of our ZeroVLM.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 56 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

26 of 56 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  2. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct
  3. 2023Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Models18 Aug

Other references · 23

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2023A Survey of Large Language Modelscited by 2,576
  8. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  9. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  10. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  11. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  12. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
11 more
  1. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  2. 2022Visual Classification via Description from Large Language Modelscited by 336
  3. 2019Visual Entailment: A Novel Task for Fine-Grained Image Understandingcited by 252
  4. 2021Visually Grounded Reasoning across Languages and Culturescited by 189
  5. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 160
  6. 2020ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Datacited by 137
  7. 2019Supervised Multimodal Bitransformers for Classifying Images and Textcited by 130
  8. 2023InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Languagecited by 96
  9. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  10. 2022VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challengescited by 14
  11. 2023From Association to Generation: Text-only Captioning by Unsupervised Cross-modal Mappingcited by 11

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  2. 2025Mitigating Hallucinations in Multimodal Spatial Relations through Constraint-Aware Promptingcited by 7
  3. 2025A Review of 3D Object Detection with Vision-Language Modelscited by 4
  4. 2025Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neurosciencecited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  2. 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
  3. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
  4. 2025G2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning
  5. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
  6. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
  7. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
  8. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas