Physically Compatible 3D Object Modeling from a Single Image
Not on the map. Why?
Abstract (arXiv)
We present a computational framework that transforms single images into 3D physical objects. The visual geometry of a physical object in an image is determined by three orthogonal attributes: mechanical properties, external forces, and rest-shape geometry. Existing single-view 3D reconstruction methods often overlook this underlying composition, presuming rigidity or neglecting external forces. Consequently, the reconstructed objects fail to withstand real-world physical forces, resulting in instability or undesirable deformation -- diverging from their intended designs as depicted in the image. Our optimization framework addresses this by embedding physical compatibility into the reconstruction process. We explicitly decompose the three physical attributes and link them through static equilibrium, which serves as a hard constraint, ensuring that the optimized physical shapes exhibit desired physical behaviors. Evaluations on a dataset collected from Objaverse demonstrate that our framework consistently enhances the physical realism of 3D models over existing methods. The utility of our framework extends to practical applications in dynamic simulations and 3D printing, where adherence to physical compatibility is paramount.
Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 28 of its 52 references that are in the map. First posted on arXiv 30 May 2024.
Builds on
28 of 52 references.
References · 28
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
- 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
16 more
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
- 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 368
- 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
- 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 344
- 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 317
- 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
- 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
- 20223D Neural Field Generation using Triplane Diffusioncited by 274
- 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
- 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 145
- 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
- 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 92
- 2023PIE-NeRF: Physics-based Interactive Elastodynamics with NeRFcited by 51
- 2024Physical Property Understanding from Language-Embedded Feature Fieldscited by 45
- 2020Modeling of Personalized Anatomy using Plastic Strainscited by 4
Built on by
Citing papers per year: 2024: 4; 2025: 23; 2026: 11.
38 papers.
Sort:
By area:
- 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
- 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 40
- 2025Generative Physical AI in Vision: A Surveycited by 24
- 2025DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundnesscited by 24
- 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
- 2025Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixelscited by 22
- 2024PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Imagecited by 19
- 2025Science-T2I: Addressing Scientific Illusions in Image Synthesiscited by 15
- 2025PAT3D: Physics-Augmented Text-to-3D Scene Generationcited by 14
- 2024GRS: Generating Robotic Simulation Tasks from Real-World Imagescited by 13
- 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Videocited by 13
- 2025Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGCcited by 11
26 more
- 2025DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videoscited by 7
- 2026PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangementcited by 7
- 2026PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objectscited by 7
- 2025Is Single-View Mesh Reconstruction Ready for Robotics?cited by 6
- 2025SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materialscited by 5
- 2025ParticleGS: Learning Neural Gaussian Particle Dynamics from Videos for Prior-free Physical Motion Extrapolationcited by 5
- 2025PhysGen: Physically Grounded 3D Shape Generation for Industrial Designcited by 5
- 2024GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objectscited by 4
- 2025MIND: Microstructure INverse Design with Generative Hybrid Neural Representationcited by 4
- 2025From 2D to 3D Cognition: A Brief Survey of General World Modelscited by 4
- 2024Neurally Integrated Finite Elements for Differentiable Elasticity on Evolving Domainscited by 3
- 2025InfiniHuman: Infinite 3D Human Creation with Precise Controlcited by 3
- 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstructioncited by 3
- 2025TopoGaussian: Inferring Internal Topology Structures from Visual Cluescited by 2
- 2025Inverse Rendering for High-Genus Surface Meshes from Multi-View Imagescited by 2
- 2025DensiCrafter: Physically-Constrained Generation and Fabrication of Self-Supporting Hollow Structurescited by 1
- 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolutioncited by 1
- 2025XDen-1K: A Density Field Dataset of Real-World Objectscited by 0
- 2025Prompt-to-Parts: Generative AI for Physical Assembly and Scalable Instructionscited by 0
- 2026Generating Physically Sound Designs from Text and a Set of Physical Constraintscited by 0
- 2026Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touchcited by 0
- 20263D Generation for Embodied AI and Robotic Simulation: A Surveycited by 0
- 2026MonoPhysics: Estimating Geometry, Appearance, and Physical Parameters from Monocular Videoscited by 0
- 2026PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assetscited by 0
- 2026SNAP3D: Physically Grounded 3D Parts for Assembly from a Single Imagecited by 0
- 2026CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Imagecited by 0
- 2026CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Image22 Sep 2026
- 2026SNAP3D: Physically Grounded 3D Parts for Assembly from a Single Image11 Sep 2026
- 2026PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assets8 Aug 2026
- 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstruction19 Jun 2026
- 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolution16 Jun 2026
- 2026MonoPhysics: Estimating Geometry, Appearance, and Physical Parameters from Monocular Videos28 May 2026
- 2026PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects20 May 2026
- 20263D Generation for Embodied AI and Robotic Simulation: A Survey29 Apr 2026
- 2026Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch10 Apr 2026
- 2026PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangement16 Feb 2026
- 2026Generating Physically Sound Designs from Text and a Set of Physical Constraints2 Feb 2026
- 2025XDen-1K: A Density Field Dataset of Real-World Objects11 Dec 2025
26 more
- 2025Prompt-to-Parts: Generative AI for Physical Assembly and Scalable Instructions10 Dec 2025
- 2025PhysGen: Physically Grounded 3D Shape Generation for Industrial Design29 Nov 2025
- 2025PAT3D: Physics-Augmented Text-to-3D Scene Generation26 Nov 2025
- 2025Inverse Rendering for High-Genus Surface Meshes from Multi-View Images24 Nov 2025
- 2025DensiCrafter: Physically-Constrained Generation and Fabrication of Self-Supporting Hollow Structures12 Nov 2025
- 2025InfiniHuman: Infinite 3D Human Creation with Precise Control13 Oct 2025
- 2025DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videos9 Oct 2025
- 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video7 Oct 2025
- 2025Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixels20 Aug 2025
- 2025PhysX-3D: Physical-Grounded 3D Asset Generation16 Jul 2025
- 2025From 2D to 3D Cognition: A Brief Survey of General World Models25 Jun 2025
- 2025ParticleGS: Learning Neural Gaussian Particle Dynamics from Videos for Prior-free Physical Motion Extrapolation26 May 2025
- 2025Is Single-View Mesh Reconstruction Ready for Robotics?23 May 2025
- 2025Generating Physically Stable and Buildable Brick Structures from Text8 May 2025
- 2025Science-T2I: Addressing Scientific Illusions in Image Synthesis17 Apr 2025
- 2025SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materials17 Apr 2025
- 2025DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundness28 Mar 2025
- 2025TopoGaussian: Inferring Internal Topology Structures from Visual Clues16 Mar 2025
- 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image18 Feb 2025
- 2025Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGC10 Feb 2025
- 2025MIND: Microstructure INverse Design with Generative Hybrid Neural Representation1 Feb 2025
- 2025Generative Physical AI in Vision: A Survey19 Jan 2025
- 2024GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objects23 Dec 2024
- 2024PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Image27 Nov 2024
- 2024GRS: Generating Robotic Simulation Tasks from Real-World Images20 Oct 2024
- 2024Neurally Integrated Finite Elements for Differentiable Elasticity on Evolving Domains12 Oct 2024
- 2025PAT3D: Physics-Augmented Text-to-3D Scene Generationcited by 14
- 2025PAT3D: Physics-Augmented Text-to-3D Scene Generation26 Nov 2025
- 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
- 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 40
- 2025Generative Physical AI in Vision: A Surveycited by 24
- 2025DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundnesscited by 24
- 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
- 2025Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixelscited by 22
- 2024PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Imagecited by 19
- 2025Science-T2I: Addressing Scientific Illusions in Image Synthesiscited by 15
- 2024GRS: Generating Robotic Simulation Tasks from Real-World Imagescited by 13
- 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Videocited by 13
- 2025Grounding Creativity in Physics: A Brief Survey of Physical Priors in AIGCcited by 11
- 2025DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videoscited by 7
12 most cited of 33 All 33 (JSON)
- 2026CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Image22 Sep 2026
- 2026SNAP3D: Physically Grounded 3D Parts for Assembly from a Single Image11 Sep 2026
- 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstruction19 Jun 2026
- 2026Adaptive Volumetric Mechanical Property Fields Invariant to Resolution16 Jun 2026
- 2026MonoPhysics: Estimating Geometry, Appearance, and Physical Parameters from Monocular Videos28 May 2026
- 2026PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects20 May 2026
- 20263D Generation for Embodied AI and Robotic Simulation: A Survey29 Apr 2026
- 2026Generating Physically Sound Designs from Text and a Set of Physical Constraints2 Feb 2026
- 2025XDen-1K: A Density Field Dataset of Real-World Objects11 Dec 2025
- 2025Prompt-to-Parts: Generative AI for Physical Assembly and Scalable Instructions10 Dec 2025
- 2025PhysGen: Physically Grounded 3D Shape Generation for Industrial Design29 Nov 2025
- 2025Inverse Rendering for High-Genus Surface Meshes from Multi-View Images24 Nov 2025
12 newest of 33 All 33 (JSON)