mlsanityData to 7 Oct 2026

MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse

arXiv 2503.18470 ↗First posted 24 Mar 2025cs.CV cs.AIcited by 50

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

We present MetaSpatial, the first reinforcement learning (RL)-based framework designed to enhance 3D spatial reasoning in vision-language models (VLMs), enabling real-time 3D scene generation without the need for hard-coded optimizations. MetaSpatial addresses two core challenges: (i) the lack of internalized 3D spatial reasoning in VLMs, which limits their ability to generate realistic layouts, and (ii) the inefficiency of traditional supervised fine-tuning (SFT) for layout generation tasks, as perfect ground truth annotations are unavailable. Our key innovation is a multi-turn RL-based optimization mechanism that integrates physics-aware constraints and rendered image evaluations, ensuring generated 3D layouts are coherent, physically plausible, and aesthetically consistent. Methodologically, MetaSpatial introduces an adaptive, iterative reasoning process, where the VLM refines spatial arrangements over multiple turns by analyzing rendered outputs, improving scene coherence progressively. Empirical evaluations demonstrate that MetaSpatial significantly enhances the spatial consistency and formatting stability of various scale models. Post-training, object placements are more realistic, aligned, and functionally coherent, validating the effectiveness of RL for 3D spatial reasoning in metaverse, AR/VR, digital twins, and game development applications. Our code, data, and training pipeline are publicly available at https://github.com/PzySeere/MetaSpatial.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 30 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

16 of 30 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space14 Mar
  2. 2025AutoSpatial: Visual-Language Reasoning for Social Robot Navigation through Efficient Spatial Reasoning Learning10 Mar
  3. 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models3 Dec

Other references · 13

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  4. 2024OpenAI o1 System Cardcited by 1,220
  5. 2024I-Design: Personalized LLM Interior Designercited by 87
  6. 2023Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraintscited by 59
  7. 2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
  8. 2024Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Modelscited by 27
  9. 2024Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Actioncited by 19
  10. 2024Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion?cited by 17
  11. 2025LAYOUTDREAMER: Physics-guided Layout for Text-to-3D Compositional Scene Generationcited by 11
  12. 2025Do Code LLMs Understand Design Patterns?cited by 9
1 more
  1. 2024Lay-A-Scene: Personalized 3D Object Arrangement Using Text-to-Image Priorscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 28; 2026: 22.

50 papers.

Sort:

By area:

  1. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  4. 2025Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoningcited by 67
  5. 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 53
  6. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  7. 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
  8. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  9. 2025Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMscited by 30
  10. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methodscited by 22
  11. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratchcited by 17
  12. 2025Direct Reasoning Optimization: Token-Level Reasoning Reflectivity Meets Rubric Gates for Unverifiable Taskscited by 16
38 more
  1. 2025OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimizationcited by 14
  2. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 13
  3. 2025Unveiling the Compositional Ability Gap in Vision-Language Reasoning Modelcited by 13
  4. 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMscited by 12
  5. 2025NP-Engine: Empowering Optimization Reasoning in Large Language Models with Verifiable Synthetic NP Problemscited by 11
  6. 2026Reinforcement Learning from Rich Feedback with Distributional DAggercited by 9
  7. 2025Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignmentcited by 8
  8. 2025Reinforcement Learning for Large Model: A Surveycited by 8
  9. 2025Yo'City: Personalized and Boundless 3D Realistic City Scene Generation via Self-Critic Expansioncited by 8
  10. 2025Vision-Language Models as Differentiable Semantic and Spatial Rewards for Text-to-3D Generationcited by 6
  11. 2026SceneReVis: A Self-Reflective Vision-Grounded Framework for 3D Indoor Scene Synthesis via Multi-turn RLcited by 6
  12. 2026When Domains Interact: Asymmetric and Order-Sensitive Cross-Domain Effects in Reinforcement Learning for Reasoningcited by 5
  13. 2025Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safetycited by 4
  14. 2025CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoningcited by 4
  15. 2025edgeVLM: Cloud-edge Collaborative Real-time VLM based on Context Transfercited by 3
  16. 2025AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learningcited by 3
  17. 2026Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editingcited by 3
  18. 2026Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editingcited by 3
  19. 2026Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Modelscited by 3
  20. 2025FairReason: Balancing Reasoning and Social Bias in MLLMscited by 2
  21. 2026Ground What You See: Hallucination-Resistant MLLMs via Caption Feedback, Diversity-Aware Sampling, and Conflict Regularizationcited by 2
  22. 2026Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Modelcited by 2
  23. 2026AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraintscited by 2
  24. 2026How Far Are Vision-Language Models from Constructing the Real World? A Benchmark for Physical Generative Reasoningcited by 2
  25. 2025Towards Secure and Explainable Smart Contract Generation with Security-Aware Group Relative Policy Optimizationcited by 1
  26. 2025MetaFind: Scene-Aware 3D Asset Retrieval for Coherent Metaverse Scene Generationcited by 1
  27. 2026XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environmentscited by 1
  28. 2026OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mappingcited by 1
  29. 2026iARCS: Iterative Agentic RL for Controllable 3D Scene Generationcited by 1
  30. 2026RL-RIG: A Generative Spatial Reasoner via Intrinsic Reflectioncited by 0
  31. 2026R-C2: Cycle-Consistent Reinforcement Learning Improves Multimodal Reasoningcited by 0
  32. 2026SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenescited by 0
  33. 2026Orchestrating Spatial Semantics via a Zone-Graph Paradigm for Intricate Indoor Scene Generationcited by 0
  34. 2026Sketch2MinSurf: Vision-Language Guided Generation of Editable Minimal Surfaces from Hand-Drawn Sketchescited by 0
  35. 2026Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Modelscited by 0
  36. 2026Function2Scene: 3D Indoor Scene Layout from Functional Specificationscited by 0
  37. 2026ThinkBLOX: 3D Indoor Scene Generation with Progressive Reasoningcited by 0
  38. 2026ScenePilot: Grow-and-Repair Policy for Text-Driven 3D Indoor Scene Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
  2. 20253D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding
  3. 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
  4. 2026SpatialStack: Layered Geometry-Language Fusion for 3D VLM Spatial Reasoning
  5. 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
  6. 2025S2-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance
  7. 2026PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learning
  8. 2026Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning