# CausalGS: Learning Physical Causality of 3D Dynamic Scenes with Gaussian Representations

arXiv: https://arxiv.org/abs/2605.10586 · HTML: https://mlsanity.com/paper/2605.10586/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 43 of its 55 references that are in the map.
- First posted on arXiv 11 May 2026.

## Builds on (most cited first, 30 of 43)

- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics](https://mlsanity.com/paper/2311.12198/) (2023)
- [Fast Dynamic Radiance Fields with Time-Aware Neural Voxels](https://mlsanity.com/paper/2205.15285/) (2022)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle](https://mlsanity.com/paper/2312.03431/) (2023)
- [Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM](https://mlsanity.com/paper/2304.14377/) (2023)
- [Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera](https://mlsanity.com/paper/2004.01294/) (2020)
- [FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding](https://mlsanity.com/paper/2312.02214/) (2023)
- [Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control](https://mlsanity.com/paper/2106.02019/) (2021)
- [GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs](https://mlsanity.com/paper/2312.00093/) (2023)
- [Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion](https://mlsanity.com/paper/2406.04338/) (2024)
- [3D-aware Image Generation using 2D Diffusion Models](https://mlsanity.com/paper/2303.17905/) (2023)
- [OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation](https://mlsanity.com/paper/2501.18982/) (2025)
- [PBNS: Physically Based Neural Simulator for Unsupervised Garment Pose Space Deformation](https://mlsanity.com/paper/2012.11310/) (2020)
- [Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data](https://mlsanity.com/paper/2206.06577/) (2022)
- [PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations](https://mlsanity.com/paper/2404.04421/) (2024)
- [GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation](https://mlsanity.com/paper/2406.14927/) (2024)
- [Reconstructing Animatable Categories from Videos](https://mlsanity.com/paper/2305.06351/) (2023)
- [DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools](https://mlsanity.com/paper/2203.17275/) (2022)
- [Generative Gaussian Splatting for Unbounded 3D City Generation](https://mlsanity.com/paper/2406.06526/) (2024)
- [PhysAnimator: Physics-Guided Generative Cartoon Animation](https://mlsanity.com/paper/2501.16550/) (2025)
- [Swift4D:Adaptive divide-and-conquer Gaussian Splatting for compact and efficient reconstruction of dynamic scene](https://mlsanity.com/paper/2503.12307/) (2025)
- [NVFi: Neural Velocity Fields for 3D Physics Learning from Dynamic Videos](https://mlsanity.com/paper/2312.06398/) (2023)
- [OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Clouds](https://mlsanity.com/paper/2210.04458/) (2022)
- [FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Video](https://mlsanity.com/paper/2503.04720/) (2025)

## Abstract (arXiv)

Learning a physical model from video data that can comprehend physical laws and predict the future trajectories of objects is a formidable challenge in artificial intelligence. Prior approaches either leverage various Partial Differential Equations (PDEs) as soft constraints in the form of PINN losses, or integrate physics simulators into neural networks; however, they often rely on strong priors or high-quality geometry reconstruction. In this paper, we propose CausalGS, a framework that learns the causal dynamics of complex dynamic 3D scenes solely from multi-view videos, while dispensing with the reliance on explicit priors. At its core is an inverse physics inference module that decouples the complex dynamics problem from the video into the joint inference of two factors: the initial velocity field representing the scene's kinematics, and the intrinsic material properties governing its dynamics. This inferred physical information is then utilized within a differentiable physics simulator to guide the learning process in a physics-regularized manner. Extensive experiments demonstrate that CausalGS surpasses the state-of-the-art on the highly challenging task of long-term future frame extrapolation, while also exhibiting advanced performance in novel view interpolation. Crucially, our work shows that, without any human annotation, the model is able to learn the complex interactions between multiple physical properties and understand the causal relationships driving the scene's dynamic evolution, solely from visual observations.