# 4D Scaffold Gaussian Splatting with Dynamic-Aware Anchor Growing for Efficient and High-Fidelity Dynamic Scene Reconstruction

arXiv: https://arxiv.org/abs/2411.17044 · HTML: https://mlsanity.com/paper/2411.17044/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Builds on (most cited first, 20 of 20)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting](https://mlsanity.com/paper/2310.10642/) (2023)
- [LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS](https://mlsanity.com/paper/2311.17245/) (2023)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis](https://mlsanity.com/paper/2312.16812/) (2023)
- [HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression](https://mlsanity.com/paper/2403.14530/) (2024)
- [EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS](https://mlsanity.com/paper/2312.04564/) (2023)
- [HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling](https://mlsanity.com/paper/2301.02238/) (2023)
- [Mixed Neural Voxels for Fast Multi-view Video Synthesis](https://mlsanity.com/paper/2212.00190/) (2022)
- [Is Attention All That NeRF Needs?](https://mlsanity.com/paper/2207.13298/) (2022)
- [MesonGS: Post-training Compression of 3D Gaussians via Efficient Attribute Transformation](https://mlsanity.com/paper/2409.09756/) (2024)
- [Implicit Gaussian Splatting with Efficient Multi-Level Tri-Plane Representation](https://mlsanity.com/paper/2408.10041/) (2024)

## Built on by (most cited first, 11 of 11)

- [Hybrid 3D-4D Gaussian Splatting for Fast Dynamic Scene Representation](https://mlsanity.com/paper/2505.13215/) (2025)
- [Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction](https://mlsanity.com/paper/2505.16533/) (2025)
- [SD-GS: Structured Deformable 3D Gaussians for Efficient Dynamic Scene Reconstruction](https://mlsanity.com/paper/2507.07465/) (2025)
- [From Tokens to Nodes: Semantic-Guided Motion Control for Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2510.02732/) (2025)
- [On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting](https://mlsanity.com/paper/2607.08250/) (2026)
- [Constrained Dynamic Gaussian Splatting](https://mlsanity.com/paper/2602.03538/) (2026)
- [GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis](https://mlsanity.com/paper/2607.21448/) (2026)
- [AdaAnchor4D: Anchor-Conditioned Spatiotemporal Feature Aggregation for Monocular UAV 4D Reconstruction](https://mlsanity.com/paper/2607.28320/) (2026)
- [TruncGradGS: Improved 3D Gaussian Splatting via Truncated Gradient Updates](https://mlsanity.com/paper/2609.03534/) (2026)
- [SplashSplat: Reconstructing Splashing Liquids from Real-World Multi-View Videos](https://mlsanity.com/paper/2609.20818/) (2026)
- [Gaussian Splatting-based Volumetric Video Compression with Sparse 4D Anchors](https://mlsanity.com/paper/2609.33969/) (2026)

## Abstract (arXiv)

Modeling dynamic scenes through 4D Gaussians offers high visual fidelity and fast rendering speeds, but comes with significant storage overhead. Recent approaches mitigate this cost by aggressively reducing the number of Gaussians. However, this inevitably removes Gaussians essential for high-quality rendering, leading to severe degradation in dynamic regions. In this paper, we introduce a novel 4D anchor-based framework that tackles the storage cost in different perspective. Rather than reducing the number of Gaussians, our method retains a sufficient quantity to accurately model dynamic contents, while compressing them into compact, grid-aligned 4D anchor features. Each anchor is processed by an MLP to spawn a set of neural 4D Gaussians, which represent a local spatiotemporal region. We design these neural 4D Gaussians to capture temporal changes with minimal parameters, making them well-suited for the MLP-based spawning. Moreover, we introduce a dynamic-aware anchor growing strategy to effectively assign additional anchors to under-reconstructed dynamic regions. Our method adjusts the accumulated gradients with Gaussians' temporal coverage, significantly improving reconstruction quality in dynamic regions. Experimental results highlight that our method achieves state-of-the-art visual quality in dynamic regions, outperforming all baselines by a large margin with practical storage costs.