mlsanityData to 14 Oct 2026

Masked Space-Time Hash Encoding for Efficient Dynamic Scene Reconstruction

arXiv 2310.17527 ↗First posted 26 Oct 2023cs.CVcited by 43

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose the Masked Space-Time Hash encoding (MSTH), a novel method for efficiently reconstructing dynamic 3D scenes from multi-view or monocular videos. Based on the observation that dynamic scenes often contain substantial static areas that result in redundancy in storage and computations, MSTH represents a dynamic scene as a weighted combination of a 3D hash encoding and a 4D hash encoding. The weights for the two components are represented by a learnable mask which is guided by an uncertainty-based objective to reflect the spatial and temporal importance of each 3D position. With this design, our method can reduce the hash collision rate by avoiding redundant queries and modifications on static areas, making it feasible to represent a large number of space-time voxels by hash tables with small size.Besides, without the requirements to fit the large numbers of temporally redundant features independently, our method is easier to optimize and converge rapidly with only twenty minutes of training for a 300-frame dynamic scene.As a result, MSTH obtains consistently better results than previous methods with only 20 minutes of training time and 130 MB of memory storage. Code is available at https://github.com/masked-spacetime-hashing/msth

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 90 of its 102 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

90 of 102 references.

References · 90

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  7. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  8. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  9. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  11. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  12. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
38 more
  1. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
  2. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  3. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  4. 2020Neural Sparse Voxel Fieldscited by 959
  5. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
  6. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
  7. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  8. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  9. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  10. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 735
  11. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 724
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  13. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  14. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  15. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 673
  16. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  17. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  18. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  19. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
  20. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
  21. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  22. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 511
  23. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 496
  24. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  25. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  26. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 468
  27. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 449
  28. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  29. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  30. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 382
  31. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  32. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 346
  33. 2021NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Imagescited by 336
  34. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  35. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  36. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  37. 2020Learned Initializations for Optimizing Coordinate-Based Neural Representationscited by 242
  38. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235

50 most cited of 90 All 90 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 15; 2025: 14; 2026: 12.

43 papers.

Sort:

By area:

  1. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 344
  2. 2024A Survey on 3D Gaussian Splattingcited by 224
  3. 2023GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesiscited by 95
  4. 2024Grid4D: 4D Decomposed Hash Encoding for High-Fidelity Dynamic Gaussian Splattingcited by 49
  5. 2024Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splattingcited by 43
  6. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  7. 20244D Gaussian Splatting with Scale-aware Residual Field and Adaptive Optimization for Real-time Rendering of Temporally Complex Dynamic Scenescited by 37
  8. 2024Compact 3D Gaussian Splatting for Static and Dynamic Radiance Fieldscited by 25
  9. 2024Segment Any 4D Gaussianscited by 20
  10. 2024MVGS: Multi-view Regulated Gaussian Splatting for Novel View Synthesiscited by 20
  11. 2024RoDUS: Robust Decomposition of Static and Dynamic Elements in Urban Scenescited by 14
  12. 2025Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modelingcited by 12
31 more
  1. 2024D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Videocited by 11
  2. 2024Dynamic 2D Gaussians: Geometrically Accurate Radiance Fields for Dynamic Objectscited by 11
  3. 2024TimeFormer: Capturing Temporal Relationships of Deformable 3D Gaussians for Robust Reconstructioncited by 9
  4. 2025Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streamingcited by 9
  5. 2025Light4GS: Lightweight Compact 4D Gaussian Splatting Generation via Context Modelcited by 8
  6. 2025DASH: 4D Hash Encoding with Self-Supervised Decomposition for Real-Time Dynamic Scene Renderingcited by 8
  7. 2024DGNS: Deformable Gaussian Splatting and Dynamic Neural Surface for Monocular Dynamic 3D Reconstructioncited by 7
  8. 2024LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Controlcited by 6
  9. 20243D Vessel Reconstruction from Sparse-View Dynamic DSA Images via Vessel Probability Guided Attenuation Learningcited by 5
  10. 2025Video4DGen: Enhancing Video and 4D Generation through Mutual Optimizationcited by 5
  11. 2025Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Fieldcited by 5
  12. 2025SpeeDe3DGS: Speedy Deformable 3D Gaussian Splatting with Temporal Pruning and Motion Groupingcited by 5
  13. 2025ImViD: Immersive Volumetric Videos for Enhanced VR Engagementcited by 3
  14. 2025Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizercited by 2
  15. 2025StreamSTGS: Streaming Spatial and Temporal Gaussian Grids for Real-Time Free-Viewpoint Videocited by 2
  16. 2026SplatWeaver: Learning to Allocate Gaussian Primitives for Generalizable Novel View Synthesiscited by 2
  17. 2026Time-Archival Camera Virtualization for Sports and Visual Performancescited by 1
  18. 2026textit4DSurf: High-Fidelity Dynamic Scene Surface Reconstructioncited by 1
  19. 2026Realizing Immersive Volumetric Video: A Multimodal Framework for 6-DoF VR Engagementcited by 1
  20. 2026On the Design of Mixture-of-Experts for Dynamic Gaussian Splattingcited by 1
  21. 2026Reconstructing the Dynamic World: A Representation-Centric View of 4D Scene Reconstructioncited by 1
  22. 2025CTRL-GS: Cascaded Temporal Residue Learning for 4D Gaussian Splattingcited by 0
  23. 2025MVG4D: Image Matrix-Based Multi-View and Motion Generation for 4D Content Creation from a Single Imagecited by 0
  24. 2025Every Camera Effect, Every Time, All at Once: 4D Gaussian Ray Tracing for Physics-based Camera Effect Data Generationcited by 0
  25. 20254D Neural Voxel Splatting: Dynamic Scene Rendering with Voxelized Guassian Splattingcited by 0
  26. 2026GS-DMSR: Dynamic Sensitive Multi-scale Manifold Enhancement for Accelerated High-Quality 3D Gaussian Splattingcited by 0
  27. 2026CloDS: Visual-Only Unsupervised Cloth Dynamics Learning in Unknown Conditionscited by 0
  28. 2026DLGStream: Dynamic Language-embedded Guassian Splatting for Open-vocabulary Enabled Free-viewpoint Video Streamingcited by 0
  29. 2026S2GS: Structured Sparse Gaussian Streaming for Efficient Free-Viewpoint Video Reconstruction on Edge-IoT Devicescited by 0
  30. 2026Towards Alias-Free 4D Gaussian Representations with Motion-Aware Filteringcited by 0
  31. 2026ATGS: Anchored Temporal Gaussian Splatting for Long Volumetric Video Representationcited by 0