NeoWorld-Pro: Programming Interactive Scenes from Monocular Images for Embodied Simulation
Not on the map. Why?
Abstract (arXiv)
The advancement of Embodied AI necessitates high-quality simulation assets that faithfully mirror the real world. However, transforming raw visual observations into simulation-ready scenes remains challenging due to the lack of physical grounding and scene-level interactivity in current image-to-URDF methods. We propose NeoWorld-Pro, a framework that reformulates monocular scene reconstruction as procedural programming for interactive 3D environments. Leveraging the zero-shot reasoning and code synthesis capabilities of MLLMs, NeoWorld-Pro converts a single RGB image into executable programs specifying object geometry, articulation, and physical properties. A physics-in-the-loop mechanism then iteratively refines the generated programs by validating their execution in a physics engine, enforcing physically plausible articulations, valid object compositions and interactions, and accurate spatial relationships. Experiments show that NeoWorld-Pro outperforms open-loop and prior monocular reconstruction methods, while enabling complex downstream tasks such as stable stacking and fine-grained manipulation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 45 references that are in the map. First posted on arXiv 25 Aug 2026.
Builds on
41 of 45 references.
References · 41
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
- 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 260
- 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 237
- 2019Category-Level Articulated Object Pose Estimationcited by 159
- 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 139
- 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 111
- 2024Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Modelcited by 96
- 2021CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Cloudscited by 84
- 2024Real2Code: Reconstruct Articulated Objects via Code Generationcited by 69
- 2024SINGAPO: Single Image Controlled Generation of Articulated Parts in Objectscited by 69
29 more
- 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AIcited by 50
- 2026Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoningcited by 49
- 2025Articulate AnyMesh: Open-Vocabulary 3D Articulated Objects Modelingcited by 43
- 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 42
- 2025PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Imagecited by 37
- 2026Articraft: An Agentic System for Scalable Articulated 3D Asset Generationcited by 30
- 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
- 2025DreamArt: Generating Interactable Articulated Objects from a Single Imagecited by 25
- 2024ArtFormer: Controllable Generation of Diverse 3D Articulated Objectscited by 22
- 2025FreeArt3D: Training-Free Articulated Object Generation using 3D Diffusioncited by 22
- 2025DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Datacited by 21
- 2025URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Modelcited by 21
- 2025Infinite Mobility: Scalable High-Fidelity Synthesis of Articulated Objects via Procedural Generationcited by 19
- 2025GaussianArt: Unified Modeling of Geometry and Motion for Articulated Objectscited by 17
- 2025LARM: A Large Articulated-Object Reconstruction Modelcited by 12
- 2024Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Descriptioncited by 9
- 2025Kinematify: Open-Vocabulary Synthesis of High-DoF Articulated Objectscited by 9
- 2025ART: Articulated Reconstruction Transformercited by 9
- 2025Guiding Diffusion-Based Articulated Object Generation by Partial Point Cloud Alignment and Physical Plausibility Constraintscited by 8
- 2026ArtLLM: Generating Articulated Assets via 3D LLMcited by 8
- 2025REACT3D: Recovering Articulations for Interactive Physical 3D Scenescited by 7
- 2025SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledgecited by 7
- 2024AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registrationcited by 5
- 2026MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstructioncited by 5
- 2026SIMART: Decomposing Monolithic Meshes into Sim-ready Articulated Assets via MLLMcited by 5
- 2025ArtiWorld: LLM-Driven Articulation of 3D Objects in Scenescited by 3
- 2026MotionAnymesh: Physics-Grounded Articulation for Simulation-Ready Digital Twinscited by 2
- 2025NeoWorld: Neural Simulation of Explorable Virtual Worlds via Progressive 3D Unfoldingcited by 1
- 2026GEAR: GEometry-motion Alternating Refinement for Articulated Object Modeling with Gaussian Splattingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects
- 2026Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
- 2026Fysiverse-3D-SimReady Technical Report: Agentic Physical Simulation for Pragmatic 3D World Reconstruction
- 2026Mirage2Matter: A Physically Grounded Gaussian World Model from Video
- 2026WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes
- 20264DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation
- 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI
- 2026CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumes