SnapPhysics: A Physics-Aware Scene Graph from a Single View for Interactive Mixed Reality Scenes
Not on the map. Why?
Abstract (arXiv)
We propose SnapPhysics, a training-free framework that reconstructs 3D objects and estimates their physical properties such as mass, friction, and center of gravity from a single image. For physically coherent interactions in mixed reality (MR), such properties are as important as geometry. Prior approaches infer them by analyzing object dynamics in video, which is computationally costly, or by querying vision-language models (VLMs) on single images, which lacks geometric grounding and inter-object relationships. We address these limitations by combining instance-level 3D reconstruction and spatial alignment with a physics-aware scene graph that encodes these relationships and per-object metric geometry as structured context for VLM-based property reasoning. Experiments on 3D-FRONT show that SnapPhysics improves scene-level F-Score by 18.6% over the best learning-based method, and on real captured scenes with ground-truth mass, it reduces the mean absolute log difference error (mALDE) by up to 20.5% and improves log-scale correlation (r2ls) by up to 19.6% over VLM-only estimation. SnapPhysics enables physically interactive MR experiences without manual parameter tuning. Project page: https://snapphysics-ismar2026.github.io/.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 50 of its 69 references that are in the map. First posted on arXiv 17 Sep 2026.
Builds on
50 of 69 references.
References · 50
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2024Depth Anything V2cited by 1,422
- 2025SAM 3: Segment Anything with Conceptscited by 906
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
- 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
- 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
- 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
- 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 281
38 more
- 2025SAM 3D: 3Dfy Anything in Imagescited by 247
- 2020Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructionscited by 208
- 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
- 2024VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Realitycited by 179
- 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 172
- 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 160
- 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 154
- 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 125
- 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 117
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
- 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 106
- 2025PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videoscited by 87
- 2021Panoptic 3D Scene Reconstruction From a Single RGB Imagecited by 70
- 2025OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generationcited by 69
- 2016Learning Physical Intuition of Block Towers by Examplecited by 68
- 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 61
- 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 54
- 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
- 2024Physical Property Understanding from Language-Embedded Feature Fieldscited by 48
- 2024PhyRecon: Physically Plausible Neural Scene Reconstructioncited by 45
- 2024REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignmentcited by 45
- 2022Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenescited by 44
- 2023Single-view 3D Scene Reconstruction with High-fidelity Shape and Texturecited by 37
- 2025PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Imagecited by 37
- 2024Efficient Physics Simulation for 3D Scenes via MLLM-Guided Gaussian Splattingcited by 32
- 2025ArtiScene: Language-Driven Artistic 3D Scene Generation Through Image Intermediarycited by 28
- 2015Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Imagescited by 26
- 2024GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMscited by 23
- 2020Retargetable AR: Context-aware Augmented Reality in Indoor Scenes based on 3D Scene Graphcited by 19
- 2025LLM Meets Scene Graph: Can Large Language Models Understand and Generate Scene Graphs? A Benchmark and Empirical Studycited by 9
- 2025ImaginateAR: AI-Assisted In-Situ Authoring in Augmented Realitycited by 8
- 2025Radiance Fields in XR: A Survey on How Radiance Fields are Envisioned and Addressed for XR Researchcited by 6
- 2025Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulationcited by 5
- 2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Modelscited by 5
- 2024Meta-Objects: Interactive and Multisensory Virtual Objects Learned from the Real World for Use in Augmented Realitycited by 4
- 2026SceneLinker: Compositional 3D Scene Generation via Semantic Scene Graph from RGB Sequencescited by 3
- 2025Perceived Weight of Mediated Reality Stickscited by 1
- 2026DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priorscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image
- 2025PhysID: Physics-based Interactive Dynamics from a Single-view Image
- 2026PhysInOne: Visual Physics Learning and Reasoning in One Suite
- 2026PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning
- 2026Occlusion-Robust Multi-Object Decoupling for Physics-Based Robotic Interaction
- 2026Beyond Static Vision: Scene Dynamic Field Unlocks Intuitive Physics Understanding in Multi-modal Large Language Models
- 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstruction
- 2026TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image