# Dynamics-Aware Gaussian Splatting Streaming Towards Fast On-the-Fly 4D Reconstruction

arXiv: https://arxiv.org/abs/2411.14847 · HTML: https://mlsanity.com/paper/2411.14847/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 74 references that are in the map.
- First posted on arXiv 22 Nov 2024.

## Builds on (most cited first, 30 of 50)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting](https://mlsanity.com/paper/2310.10642/) (2023)
- [LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS](https://mlsanity.com/paper/2311.17245/) (2023)
- [Gaussian Grouping: Segment and Edit Anything in 3D Scenes](https://mlsanity.com/paper/2312.00732/) (2023)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis](https://mlsanity.com/paper/2312.16812/) (2023)
- [GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting](https://mlsanity.com/paper/2311.14521/) (2023)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [3D Gaussian Splatting as Markov Chain Monte Carlo](https://mlsanity.com/paper/2404.09591/) (2024)
- [HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression](https://mlsanity.com/paper/2403.14530/) (2024)
- [Tracking Anything with Decoupled Video Segmentation](https://mlsanity.com/paper/2309.03903/) (2023)
- [4D-Rotor Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes](https://mlsanity.com/paper/2402.03307/) (2024)
- [HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling](https://mlsanity.com/paper/2301.02238/) (2023)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)

## Built on by (most cited first, 14 of 14)

- [Spatia: Video Generation with Updatable Spatial Memory](https://mlsanity.com/paper/2512.15716/) (2025)
- [Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming](https://mlsanity.com/paper/2503.08166/) (2025)
- [Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos](https://mlsanity.com/paper/2509.24209/) (2025)
- [RemedyGS: Defend 3D Gaussian Splatting against Computation Cost Attacks](https://mlsanity.com/paper/2511.22147/) (2025)
- [Flash-VAED: Plug-and-Play VAE Decoders for Efficient Video Generation](https://mlsanity.com/paper/2602.19161/) (2026)
- [Feed-Forward 3D Gaussian Splatting Compression with Long-Context Modeling](https://mlsanity.com/paper/2512.00877/) (2025)
- [Points-to-3D: Structure-Aware 3D Generation with Point Cloud Priors](https://mlsanity.com/paper/2603.18782/) (2026)
- [SUCCESS-GS: Survey of Compactness and Compression for Efficient Static and Dynamic Gaussian Splatting](https://mlsanity.com/paper/2512.07197/) (2025)
- [Real-Time Human Frontal View Synthesis from a Single Image](https://mlsanity.com/paper/2603.15433/) (2026)
- [Mon3tr: Monocular 3D Telepresence with Pre-built Gaussian Avatars as Amortization](https://mlsanity.com/paper/2601.07518/) (2026)
- [On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting](https://mlsanity.com/paper/2607.08250/) (2026)
- [Multi4D: High-Fidelity Dynamic Gaussian Splatting via Multi-Level Competitive Allocation](https://mlsanity.com/paper/2606.22197/) (2026)
- [CC-4DGS: Computational Deformation and Point-Cloud Compression for Storage-Efficient Dynamic Gaussian Splatting](https://mlsanity.com/paper/2609.02184/) (2026)
- [Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras](https://mlsanity.com/paper/2609.15032/) (2026)

## Abstract (arXiv)

The recent development of 3D Gaussian Splatting (3DGS) has led to great interest in 4D dynamic spatial reconstruction. Existing approaches mainly rely on full-length multi-view videos, while there has been limited exploration of online reconstruction methods that enable on-the-fly training and per-timestep streaming. Current 3DGS-based streaming methods treat the Gaussian primitives uniformly and constantly renew the densified Gaussians, thereby overlooking the difference between dynamic and static features as well as neglecting the temporal continuity in the scene. To address these limitations, we propose a novel three-stage pipeline for iterative streamable 4D dynamic spatial reconstruction. Our pipeline comprises a selective inheritance stage to preserve temporal continuity, a dynamics-aware shift stage to distinguish dynamic and static primitives and optimize their movements, and an error-guided densification stage to accommodate emerging objects. Our method achieves state-of-the-art performance in online 4D reconstruction, demonstrating the fastest on-the-fly training, superior representation quality, and real-time rendering capability. Project page: https://www.liuzhening.top/DASS