# GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs

arXiv: https://arxiv.org/abs/2412.11258 · HTML: https://mlsanity.com/paper/2412.11258/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 57 references that are in the map.
- First posted on arXiv 15 Dec 2024.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [LangSplat: 3D Language Gaussian Splatting](https://mlsanity.com/paper/2312.16084/) (2023)
- [PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics](https://mlsanity.com/paper/2311.12198/) (2023)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [MVImgNet: A Large-scale Dataset of Multi-view Images](https://mlsanity.com/paper/2303.06042/) (2023)
- [NeRF-Editing: Geometry Editing of Neural Radiance Fields](https://mlsanity.com/paper/2205.04978/) (2022)
- [Scalable 3D Captioning with Pretrained Models](https://mlsanity.com/paper/2306.07279/) (2023)
- [PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification](https://mlsanity.com/paper/2303.05512/) (2023)
- [GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation](https://mlsanity.com/paper/2401.04092/) (2024)
- [Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes](https://mlsanity.com/paper/2308.08769/) (2023)
- [Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians](https://mlsanity.com/paper/2403.09434/) (2024)
- [NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos](https://mlsanity.com/paper/2210.12352/) (2022)
- [GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation](https://mlsanity.com/paper/2406.14927/) (2024)
- [Physical Property Understanding from Language-Embedded Feature Fields](https://mlsanity.com/paper/2404.04242/) (2024)
- [Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing](https://mlsanity.com/paper/2404.01223/) (2024)
- [Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modeling](https://mlsanity.com/paper/2410.18912/) (2024)
- [Soft Biomimetic Optical Tactile Sensing with the TacTip: A Review](https://mlsanity.com/paper/2105.14455/) (2021)
- [Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Maps](https://mlsanity.com/paper/2406.18115/) (2024)
- [Neural Impostor: Editing Neural Radiance Fields with Explicit Shape Manipulation](https://mlsanity.com/paper/2310.05391/) (2023)

## Built on by (most cited first, 23 of 23)

- [Exploring the Evolution of Physics Cognition in Video Generation: A Survey](https://mlsanity.com/paper/2503.21765/) (2025)
- [PhysGS: Bayesian-Inferred Gaussian Splatting for Physical Property Estimation](https://mlsanity.com/paper/2511.18570/) (2025)
- [MVInverse: Feed-forward Multi-view Inverse Rendering in Seconds](https://mlsanity.com/paper/2512.21003/) (2025)
- [ContraGS: Codebook-Condensed and Trainable Gaussian Splatting for Fast, Memory-Efficient Reconstruction](https://mlsanity.com/paper/2509.03775/) (2025)
- [NeuROK: Generative 4D Neural Object Kinematics](https://mlsanity.com/paper/2605.30347/) (2026)
- [FieryGS: In-the-Wild Fire Synthesis with Physics-Integrated Gaussian Splatting](https://mlsanity.com/paper/2605.00177/) (2026)
- [UniPhysGen: Unified Physical Grounding for Simulation-Ready 3D Assets](https://mlsanity.com/paper/2607.13586/) (2026)
- [SiPhy: Single-Image Physical Property Reasoning](https://mlsanity.com/paper/2607.22355/) (2026)
- [Building Explicit World Model for Zero-Shot Open-World Object Manipulation](https://mlsanity.com/paper/2603.13825/) (2026)
- [3D Generation for Embodied AI and Robotic Simulation: A Survey](https://mlsanity.com/paper/2604.26509/) (2026)
- [DeGS: A Scalable 3DGS Architecture via Decoupled Workload Parsing and Reorganization](https://mlsanity.com/paper/2608.02099/) (2026)
- [RoboPearls: Editable Video Simulation for Robot Manipulation](https://mlsanity.com/paper/2506.22756/) (2025)
- [A Study of the Framework and Real-World Applications of Language Embedding for 3D Scene Understanding](https://mlsanity.com/paper/2508.05064/) (2025)
- [XDen-1K: A Density Field Dataset of Real-World Objects](https://mlsanity.com/paper/2512.10668/) (2025)
- [VeloGauss: Learning Physically Consistent Gaussian Velocity Fields from Videos](https://mlsanity.com/paper/2605.10567/) (2026)
- [KnockGS:interaction-Grounded Calibrationof Physical Gaussian Representations](https://mlsanity.com/paper/2608.27365/) (2026)
- [Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videos](https://mlsanity.com/paper/2608.31025/) (2026)
- [PhysMAS: Physics-Grounded Multi-Agent Synthesis of Compositional 4D Gaussians](https://mlsanity.com/paper/2609.07174/) (2026)
- [PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Image](https://mlsanity.com/paper/2609.18920/) (2026)
- [SnapPhysics: A Physics-Aware Scene Graph from a Single View for Interactive Mixed Reality Scenes](https://mlsanity.com/paper/2609.19815/) (2026)
- [PhysForge: Material-Controllable 3D Reconstruction with Separable Components](https://mlsanity.com/paper/2610.13681/) (2026)
- [Physical-Field-Bridged Material Parameter Inference from Multi-View Deformation Videos](https://mlsanity.com/paper/2610.17591/) (2026)
- [Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation](https://mlsanity.com/paper/2610.17651/) (2026)

## Abstract (arXiv)

Estimating physical properties for visual data is a crucial task in computer vision, graphics, and robotics, underpinning applications such as augmented reality, physical simulation, and robotic grasping. However, this area remains under-explored due to the inherent ambiguities in physical property estimation. To address these challenges, we introduce GaussianProperty, a training-free framework that assigns physical properties of materials to 3D Gaussians. Specifically, we integrate the segmentation capability of SAM with the recognition capability of GPT-4V(ision) to formulate a global-local physical property reasoning module for 2D images. Then we project the physical properties from multi-view 2D images to 3D Gaussians using a voting strategy. We demonstrate that 3D Gaussians with physical property annotations enable applications in physics-based dynamic simulation and robotic grasping. For physics-based dynamic simulation, we leverage the Material Point Method (MPM) for realistic dynamic simulation. For robot grasping, we develop a grasping force prediction strategy that estimates a safe force range required for object grasping based on the estimated physical properties. Extensive experiments on material segmentation, physics-based dynamic simulation, and robotic grasping validate the effectiveness of our proposed method, highlighting its crucial role in understanding physical properties from visual data. Online demo, code, more cases and annotated datasets are available on hrefhttps://Gaussian-Property.github.iothis https URL.