# GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis

arXiv: https://arxiv.org/abs/2312.11458 · HTML: https://mlsanity.com/paper/2312.11458/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 74 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 30 of 53)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting](https://mlsanity.com/paper/2310.10642/) (2023)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Fast Dynamic Radiance Fields with Time-Aware Neural Voxels](https://mlsanity.com/paper/2205.15285/) (2022)
- [Neural Point-Based Graphics](https://mlsanity.com/paper/1906.08240/) (2019)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [DynIBaR: Neural Dynamic Image-Based Rendering](https://mlsanity.com/paper/2211.11082/) (2022)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields](https://mlsanity.com/paper/2106.13228/) (2021)
- [Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle](https://mlsanity.com/paper/2312.03431/) (2023)
- [Robust Dynamic Radiance Fields](https://mlsanity.com/paper/2301.02239/) (2023)

## Built on by (most cited first, 30 of 95)

- [A Survey on 3D Gaussian Splatting](https://mlsanity.com/paper/2401.03890/) (2024)
- [MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds](https://mlsanity.com/paper/2405.17421/) (2024)
- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting](https://mlsanity.com/paper/2404.03613/) (2024)
- [3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities](https://mlsanity.com/paper/2407.17418/) (2024)
- [SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer](https://mlsanity.com/paper/2404.03736/) (2024)
- [FreeTimeGS: Free Gaussian Primitives at Anytime and Anywhere for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2506.05348/) (2025)
- [MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors](https://mlsanity.com/paper/2406.00434/) (2024)
- [SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Video](https://mlsanity.com/paper/2412.09982/) (2024)
- [Swift4D:Adaptive divide-and-conquer Gaussian Splatting for compact and efficient reconstruction of dynamic scene](https://mlsanity.com/paper/2503.12307/) (2025)
- [Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Rendering](https://mlsanity.com/paper/2401.15318/) (2024)
- [WorDepth: Variational Language Prior for Monocular Depth Estimation](https://mlsanity.com/paper/2404.03635/) (2024)
- [MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2501.03714/) (2025)
- [SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes](https://mlsanity.com/paper/2410.17249/) (2024)
- [StreetSurfGS: Scalable Urban Street Surface Reconstruction with Planar-based Gaussian Splatting](https://mlsanity.com/paper/2410.04354/) (2024)
- [COB-GS: Clear Object Boundaries in 3DGS Segmentation Based on Boundary-Adaptive Gaussian Splitting](https://mlsanity.com/paper/2503.19443/) (2025)
- [MonoFusion: Sparse-View 4D Reconstruction via Monocular Fusion](https://mlsanity.com/paper/2507.23782/) (2025)
- [Robust 3D Gaussian Splatting for Novel View Synthesis in Presence of Distractors](https://mlsanity.com/paper/2408.11697/) (2024)
- [4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives](https://mlsanity.com/paper/2412.20720/) (2024)
- [4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time](https://mlsanity.com/paper/2506.18890/) (2025)
- [Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile Gaussian Feature Fields](https://mlsanity.com/paper/2503.20776/) (2025)
- [MoBGS: Motion Deblurring Dynamic 3D Gaussian Splatting for Blurry Monocular Video](https://mlsanity.com/paper/2504.15122/) (2025)
- [HAIF-GS: Hierarchical and Induced Flow-Guided Gaussian Splatting for Dynamic Scene](https://mlsanity.com/paper/2506.09518/) (2025)
- [GaussianUpdate: Continual 3D Gaussian Splatting Update for Changing Environments](https://mlsanity.com/paper/2508.08867/) (2025)
- [Modeling Ambient Scene Dynamics for Free-view Synthesis](https://mlsanity.com/paper/2406.09395/) (2024)
- [Few-shot Novel View Synthesis using Depth Aware 3D Gaussian Splatting](https://mlsanity.com/paper/2410.11080/) (2024)
- [Not All Frame Features Are Equal: Video-to-4D Generation via Decoupling Dynamic-Static Features](https://mlsanity.com/paper/2502.08377/) (2025)
- [DynaGSLAM: Real-Time Gaussian-Splatting SLAM for Online Rendering, Tracking, Motion Predictions of Moving Objects in Dynamic Scenes](https://mlsanity.com/paper/2503.11979/) (2025)
- [VeGaS: Video Gaussian Splatting](https://mlsanity.com/paper/2411.11024/) (2024)
- [Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming](https://mlsanity.com/paper/2503.08166/) (2025)

## Abstract (arXiv)

We propose a method that achieves state-of-the-art rendering quality and efficiency on monocular dynamic scene reconstruction using deformable 3D Gaussians. Implicit deformable representations commonly model motion with a canonical space and time-dependent backward-warping deformation field. Our method, GauFRe, uses a forward-warping deformation to explicitly model non-rigid transformations of scene geometry. Specifically, we propose a template set of 3D Gaussians residing in a canonical space, and a time-dependent forward-warping deformation field to model dynamic objects. Additionally, we tailor a 3D Gaussian-specific static component supported by an inductive bias-aware initialization approach which allows the deformation field to focus on moving scene regions, improving the rendering of complex real-world motion. The differentiable pipeline is optimized end-to-end with a self-supervised rendering loss. Experiments show our method achieves competitive results and higher efficiency than both previous state-of-the-art NeRF and Gaussian-based methods. For real-world scenes, GauFRe can train in ~20 mins and offer 96 FPS real-time rendering on an RTX 3090 GPU. Project website: https://lynl7130.github.io/gaufre/index.html