mlsanityData to 14 Oct 2026

VoxRep: Enhancing 3D Spatial Understanding in 2D Vision-Language Models via Voxel Representation

arXiv 2503.21214 ↗First posted 27 Mar 2025cs.CV cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Comprehending 3D environments is vital for intelligent systems in domains like robotics and autonomous navigation. Voxel grids offer a structured representation of 3D space, but extracting high-level semantic meaning remains challenging. This paper proposes a novel approach utilizing a Vision-Language Model (VLM) to extract "voxel semantics"-object identity, color, and location-from voxel data. Critically, instead of employing complex 3D networks, our method processes the voxel space by systematically slicing it along a primary axis (e.g., the Z-axis, analogous to CT scan slices). These 2D slices are then formatted and sequentially fed into the image encoder of a standard VLM. The model learns to aggregate information across slices and correlate spatial patterns with semantic concepts provided by the language component. This slice-based strategy aims to leverage the power of pre-trained 2D VLMs for efficient 3D semantic understanding directly from voxel representations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 20 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

10 of 20 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs17 Mar
  2. 2024LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences2 Dec
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 7

  1. 2025Gemma 3 Technical Reportcited by 1,824
  2. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  3. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  4. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  5. 2024SceneGPT: A Language Model for 3D Scene Understandingcited by 12
  6. 2025Foundational Models for 3D Point Clouds: A Survey and Outlookcited by 12
  7. 2024Hybrid Voxel Formats for Efficient Ray Tracingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Models
  2. 2025SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models
  3. 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models
  4. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D
  5. 2025G2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning
  6. 2024SpatialBot: Precise Spatial Understanding with Vision Language Models
  7. 2026Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?
  8. 2026Occ-VLM: Occupancy Grounded Vision Language Model for Indoor Scene Understanding