mlsanityData to 7 Oct 2026

DGD: Dynamic 3D Gaussians Distillation

arXiv 2405.19321 ↗First posted 29 May 2024cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

We tackle the task of learning dynamic 3D semantic radiance fields given a single monocular video as input. Our learned semantic radiance field captures per-point semantics as well as color and geometric properties for a dynamic 3D scene, enabling the generation of novel views and their corresponding semantics. This enables the segmentation and tracking of a diverse set of 3D semantic entities, specified using a simple and intuitive interface that includes a user click or a text prompt. To this end, we present DGD, a unified 3D representation for both the appearance and semantics of a dynamic 3D scene, building upon the recently proposed dynamic 3D Gaussians representation. Our representation is optimized over time with both color and semantic information. Key to our method is the joint optimization of the appearance and semantic attributes, which jointly affect the geometric properties of the scene. We evaluate our approach in its ability to enable dense semantic 3D object tracking and demonstrate high-quality results that are fast to render, for a diverse set of scenes. Our project webpage is available on https://isaaclabe.github.io/DGD-Website/

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 48 of its 54 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

48 of 54 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Segment Anythingcited by 8,158
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  10. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  12. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
36 more
  1. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  2. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  3. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  4. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  5. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  6. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  7. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  8. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  9. 2020Convolutional Occupancy Networkscited by 760
  10. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 753
  11. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  12. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  13. 2022Language-driven Semantic Segmentationcited by 711
  14. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 569
  15. 2023LERF: Language Embedded Radiance Fieldscited by 538
  16. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 470
  17. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  18. 2023LangSplat: 3D Language Gaussian Splattingcited by 410
  19. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  20. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 361
  21. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  22. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  23. 2019MOTS: Multi-Object Tracking and Segmentationcited by 222
  24. 2023Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particlecited by 210
  25. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  26. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  27. 2020Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Cameracited by 166
  28. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 136
  29. 2024FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understandingcited by 86
  30. 2023A Compact Dynamic 3D Gaussian Representation for Real-Time Dynamic View Synthesiscited by 59
  31. 2022EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Pointscited by 30
  32. 2023Polynomial Neural Fields for Subband Decomposition and Manipulationcited by 22
  33. 2022Volumetric Disentanglement for 3D Scene Manipulationcited by 13
  34. 2023Semantic Attention Flow Fields for Monocular Dynamic Scene Decompositioncited by 12
  35. 2024Semantic Flow: Learning Semantic Field of Dynamic Scenes from Monocular Videoscited by 6
  36. 2023Factored Neural Representation for Scene Understandingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 15.

23 papers.

Sort:

By area:

  1. 2025FreeTimeGS: Free Gaussian Primitives at Anytime and Anywhere for Dynamic Scene Reconstructioncited by 53
  2. 20254D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Modelscited by 35
  3. 2025VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Predictioncited by 9
  4. 2025RePerformer: Immersive Human-centric Volumetric Videos from Playback to Photoreal Reperformancecited by 8
  5. 2025Segment then Splat: Unified 3D Open-Vocabulary Segmentation via Gaussian Splattingcited by 7
  6. 2025SurgTPGS: Semantic 3D Surgical Scene Understanding with Text Promptable Gaussian Splattingcited by 7
  7. 2025EgoSplat: Open-Vocabulary Egocentric Scene Understanding with Language Embedded 3D Gaussian Splattingcited by 4
  8. 2026SharpTimeGS: Sharp and Stable Dynamic Gaussian Splatting via Lifespan Modulationcited by 1
  9. 2026Splat and Distill: Augmenting Teachers with Feed-Forward 3D Reconstruction For 3D-Aware Distillationcited by 1
  10. 20264D Synchronized Fields: Motion-Language Gaussian Splatting for Temporal Scene Understandingcited by 1
  11. 2026PanopticQuery: Unified Query-Time Reasoning for 4D Scenescited by 1
  12. 2025Rad-GS: Radar-Vision Integration for 3D Gaussian Splatting SLAM in Outdoor Environmentscited by 0
11 more
  1. 2026TIBR4D: Tracing-Guided Iterative Boundary Refinement for Efficient 4D Gaussian Segmentationcited by 0
  2. 2026RetimeGS: Continuous-Time Reconstruction of 4D Gaussian Splattingcited by 0
  3. 2026Inst4DGS: Instance-Decomposed 4D Gaussian Splatting with Multi-Video Label Permutation Learningcited by 0
  4. 2026FreeTimeGS++: Secrets of Dynamic Gaussian Splatting and Their Principlescited by 0
  5. 2026Do Gaussian Scenes Contain Enough Structure for Intrinsic Segmentation?cited by 0
  6. 2026Multi4D: High-Fidelity Dynamic Gaussian Splatting via Multi-Level Competitive Allocationcited by 0
  7. 2026DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splattingcited by 0
  8. 2026RoofGS: Roofline-Guided End-to-End Acceleration of 3D Gaussian Splattingcited by 0
  9. 2026SMG: Semantic Motion Graph for Monocular Dynamic Gaussian Splattingcited by 0
  10. 2026UniFusion: Sparse-View 4D Reconstruction via Unified Spatio-temporal Depth Alignmentcited by 0
  11. 2026ProDyGS: Dynamic Gaussian Splatting from a Single Static Monocular Cameracited by 0