mlsanityData to 14 Oct 2026

Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction

arXiv 2309.13101 ↗First posted 22 Sep 2023cs.CVcited by 759

Not on the map. Why?

Cited by 28 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Implicit neural representation has paved the way for new approaches to dynamic scene reconstruction and rendering. Nonetheless, cutting-edge dynamic neural rendering methods rely heavily on these implicit representations, which frequently struggle to capture the intricate details of objects in the scene. Furthermore, implicit methods have difficulty achieving real-time rendering in general dynamic scenes, limiting their use in a variety of tasks. To address the issues, we propose a deformable 3D Gaussians Splatting method that reconstructs scenes using 3D Gaussians and learns them in canonical space with a deformation field to model monocular dynamic scenes. We also introduce an annealing smoothing training mechanism with no extra overhead, which can mitigate the impact of inaccurate poses on the smoothness of time interpolation tasks in real-world datasets. Through a differential Gaussian rasterizer, the deformable 3D Gaussians not only achieve higher rendering quality but also real-time rendering speed. Experiments show that our method outperforms existing methods significantly in terms of both rendering quality and speed, making it well-suited for tasks such as novel-view synthesis, time interpolation, and real-time rendering.

Key facts. Cited by 759 papers in the map (data through 14 Oct 2026). Builds on 47 of its 62 references that are in the map. First posted on arXiv 22 Sep 2023.

Builds on

47 of 62 references.

References · 47

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  4. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  5. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  7. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  8. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  9. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  10. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  11. 2020Neural Sparse Voxel Fieldscited by 959
  12. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
35 more
  1. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  2. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 737
  3. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 735
  4. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 673
  5. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  6. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  7. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  8. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 511
  9. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  10. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 468
  11. 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 467
  12. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 449
  13. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  14. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 346
  15. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 311
  16. 2023FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularizationcited by 311
  17. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 217
  18. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  19. 2022Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-timecited by 165
  20. 2023NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Imagescited by 163
  21. 2021Efficient Neural Radiance Fields for Interactive Free-viewpoint Videocited by 150
  22. 2023Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fieldscited by 143
  23. 2022StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learningcited by 140
  24. 2023F2-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectoriescited by 118
  25. 2023NeRF-DS: Neural Radiance Fields for Dynamic Specular Objectscited by 111
  26. 2022DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenescited by 103
  27. 2022Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstructioncited by 94
  28. 2021T"oRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesiscited by 88
  29. 2021Neural Trajectory Fields for Dynamic Novel View Synthesiscited by 86
  30. 2022Differentiable Point-Based Radiance Fields for Efficient View Synthesiscited by 86
  31. 2022BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fieldscited by 80
  32. 2023Representing Volumetric Videos as Dynamic MLP Mapscited by 51
  33. 2022Approximate Differentiable Rendering with Algebraic Surfacescited by 44
  34. 2023Flexible Techniques for Differentiable Rendering with 3D Gaussianscited by 28
  35. 2023Learning a Room with the Occ-SDF Hybrid: Signed Distance Function Mingled with Occupancy Aids Scene Representationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 48; 2024: 199; 2025: 276; 2026: 236.

759 papers.

Sort:

By area:

  1. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  2. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 476
  3. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  4. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 393
  5. 2023SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAMcited by 388
  6. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
  7. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 360
  8. 2023SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenescited by 347
  9. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 344
  10. 2023GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splattingcited by 335
  11. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  12. 2023GS-SLAM: Dense Visual SLAM with 3D Gaussian Splattingcited by 278
38 more
  1. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 250
  2. 2024Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussianscited by 230
  3. 2024A Survey on 3D Gaussian Splattingcited by 224
  4. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 221
  5. 20243D Gaussian Splatting as Markov Chain Monte Carlocited by 217
  6. 2023Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particlecited by 211
  7. 2024VastGaussian: Vast 3D Gaussians for Large Scene Reconstructioncited by 209
  8. 20233DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splattingcited by 195
  9. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 194
  10. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
  11. 20244D-Rotor Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenescited by 189
  12. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 179
  13. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  14. 2023GaussianEditor: Editing 3D Gaussians Delicately with Text Instructionscited by 168
  15. 2023GauHuman: Articulated Gaussian Splatting from Monocular Human Videoscited by 168
  16. 2024OmniRe: Omni Urban Scene Reconstructioncited by 161
  17. 20243DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videoscited by 160
  18. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  19. 2023Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splattingcited by 154
  20. 2024OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understandingcited by 145
  21. 2023Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Renderingcited by 143
  22. 2024HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splattingcited by 142
  23. 2024Recent Advances in 3D Gaussian Splattingcited by 131
  24. 2023DynMF: Neural Motion Factorization for Real-time Dynamic View Synthesis with 3D Gaussian Splattingcited by 124
  25. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  26. 2023GART: Gaussian Articulated Template Modelscited by 121
  27. 2023Drivable 3D Gaussian Avatarscited by 120
  28. 2024Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splattingcited by 113
  29. 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 108
  30. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 108
  31. 2023Human Gaussian Splatting: Real-time Rendering of Animatable Avatarscited by 105
  32. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  33. 20243D Gaussian as a New Era: A Surveycited by 104
  34. 2024Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splattingcited by 104
  35. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  36. 2024Motion-aware 3D Gaussian Splatting for Efficient Dynamic Scene Reconstructioncited by 100
  37. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 100
  38. 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 97

50 most cited of 759 All 759 (JSON)