Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoning
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Generative models have advanced image-conditioned 3D content creation, yet generating controllable and executable 3D scenes from a single image remains challenging. Existing 3D generative approaches can synthesize visually plausible objects and scenes, but their spatial layout estimation is coupled with specific asset generators. They struggle to jointly model object semantics, metric geometry, and scene-level spatial relationships, which are essential for interactive editing, physical simulation, and embodied applications. We propose Fysiverse-3D-Vision, a unified vision-language-geometry framework for generative 3D scene reconstruction and executable asset construction from a single image. We establish a shared representation where spatial reasoning and geometric reconstruction mutually enhance each other, allowing object layouts to be inferred beyond the constraints of individual asset generators. Our model integrates textual supervision, semantic visual cues, and geometric representations within a unified Transformer to capture scene context, metric geometry, and object-level interactions. An object-conditioned layout module performs cross-attention between target object representations and global geometric features to predict object translation, rotation, and scale. Training progressively learns geometry-language alignment, introduces layout reasoning while preserving reconstruction capability, and refines physical consistency through collision-aware optimization. By separating spatial layout reasoning from asset synthesis, Fysiverse-3D-Vision provides an adaptable interface for interactive scene editing, object-level manipulations, and executable 3D content generation. Experiments demonstrate that our framework achieves superior geometric consistency, layout estimation, rendering quality, and physical property understanding compared with existing approaches.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 59 of its 64 references that are in the map. First posted on arXiv 22 Sep 2026.
Builds on
59 of 64 references.
Closest ancestors · 3
Same direction, newest first
- 2026ProFocus: Proactive Perception and Focused Reasoning in Vision-and-Language Navigation1 Mar
- 2025G2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning26 Nov
- 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models3 Dec
Other references · 56
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2015Fast R-CNNcited by 2,930
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
- 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
- 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
- 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
38 more
- 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2025SAM 3D: 3Dfy Anything in Imagescited by 230
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
- 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 183
- 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
- 2025Native and Compact Structured Latents for 3D Generationcited by 157
- 2026Qwen3.5-Omni Technical Reportcited by 146
- 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
- 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 111
- 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
- 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
- 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 78
- 2024SINGAPO: Single Image Controlled Generation of Articulated Parts in Objectscited by 66
- 2025Ming-Omni: A Unified Multimodal Model for Perception and Generationcited by 62
- 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 60
- 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 59
- 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
- 2025Efficient Part-level 3D Object Generation via Dual Volume Packingcited by 47
- 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 40
- 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AIcited by 40
- 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 34
- 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
- 2025PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Imagecited by 32
- 2025DreamArt: Generating Interactable Articulated Objects from a Single Imagecited by 25
- 2025DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusioncited by 23
- 2025LongCat-Flash-Omni Technical Reportcited by 23
- 2025URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Modelcited by 20
- 2025Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understandingcited by 18
- 2022Restoring Vision in Hazy Weather with Hierarchical Contrastive Learningcited by 17
- 2025SplitFlow: Flow Decomposition for Inversion-Free Text-to-Image Editingcited by 15
- 2025Omni-View: Unlocking How Generation Facilitates Understanding in Unified 3D Model based on Multiview imagescited by 15
- 2025SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Modelcited by 15
- 2025UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encodingcited by 9
- 20263D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Imagecited by 9
- 20253DPR: Single Image 3D Portrait Relight using Generative Priorscited by 8
50 most cited of 56 All 56 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Fysiverse-3D-SimReady Technical Report: Agentic Physical Simulation for Pragmatic 3D World Reconstruction
- 2026SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
- 2026Flame3D: Zero-shot Compositional Reasoning of 3D Scenes with Agentic Language Models
- 2025Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling
- 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors
- 2026Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?
- 20263D Primitives are a Spatial Language for VLMs
- 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models