# Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splatting

arXiv: https://arxiv.org/abs/2503.16979 · HTML: https://mlsanity.com/paper/2503.16979/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 77 of its 86 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Builds on (most cited first, 30 of 77)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation](https://mlsanity.com/paper/2402.05054/) (2024)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting](https://mlsanity.com/paper/2310.10642/) (2023)

## Built on by (most cited first, 30 of 36)

- [LocalDyGS: Multi-view Global Dynamic Scene Modeling via Adaptive Local Implicit Feature Decoupling](https://mlsanity.com/paper/2507.02363/) (2025)
- [LongSplat: Online Generalizable 3D Gaussian Splatting from Long Sequence Images](https://mlsanity.com/paper/2507.16144/) (2025)
- [ReCon-GS: Continuum-Preserved Gaussian Streaming for Fast and Compact Reconstruction of Dynamic Scenes](https://mlsanity.com/paper/2509.24325/) (2025)
- [D-FCGS: Feedforward Compression of Dynamic Gaussian Splatting for Free-Viewpoint Videos](https://mlsanity.com/paper/2507.05859/) (2025)
- [Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos](https://mlsanity.com/paper/2509.24209/) (2025)
- [Motion Matters: Compact Gaussian Streaming for Free-Viewpoint Video Reconstruction](https://mlsanity.com/paper/2505.16533/) (2025)
- [EA3D: Online Open-World 3D Object Extraction from Streaming Videos](https://mlsanity.com/paper/2510.25146/) (2025)
- [SLARM: Streaming and Language-Aligned Reconstruction Model for Dynamic Scenes](https://mlsanity.com/paper/2603.22893/) (2026)
- [ClipGStream: Clip-Stream Gaussian Splatting for Any Length and Any Motion Multi-View Dynamic Scene Reconstruction](https://mlsanity.com/paper/2604.13746/) (2026)
- [Scale-GS: Efficient Scalable Gaussian Splatting via Redundancy-filtering Training on Streaming Content](https://mlsanity.com/paper/2508.21444/) (2025)
- [LoD-Structured 3D Gaussian Splatting for Streaming Video Reconstruction](https://mlsanity.com/paper/2601.18475/) (2026)
- [MoRGS: Efficient Per-Gaussian Motion Reasoning for Streamable Dynamic 3D Scenes](https://mlsanity.com/paper/2603.25042/) (2026)
- [Adaptive 3D Gaussian Splatting Video Streaming: Visual Saliency-Aware Tiling and Meta-Learning-Based Bitrate Adaptation](https://mlsanity.com/paper/2507.14454/) (2025)
- [Gaussian Splatting-based Low-Rank Tensor Representation for Multi-Dimensional Image Recovery](https://mlsanity.com/paper/2511.14270/) (2025)
- [Do Flat Minima Improve Sparse Novel View Synthesis?](https://mlsanity.com/paper/2511.17918/) (2025)
- [SV-GS: Sparse View 4D Reconstruction with Skeleton-Driven Gaussian Splatting](https://mlsanity.com/paper/2601.00285/) (2026)
- [Hybrid Foveated Path Tracing with Peripheral Gaussians for Immersive Anatomy](https://mlsanity.com/paper/2601.22026/) (2026)
- [textit4DSurf: High-Fidelity Dynamic Scene Surface Reconstruction](https://mlsanity.com/paper/2603.28064/) (2026)
- [Reconstructing the Dynamic World: A Representation-Centric View of 4D Scene Reconstruction](https://mlsanity.com/paper/2609.39960/) (2026)
- [Task-Oriented Communications for 3D Scene Representation: Balancing Timeliness and Fidelity](https://mlsanity.com/paper/2509.17282/) (2025)
- [Endo-G2T: Geometry-Guided & Temporally Aware Time-Embedded 4DGS For Endoscopic Scenes](https://mlsanity.com/paper/2511.21367/) (2025)
- [Does it matter which Gaussians you pick in 4D Gaussian streaming?](https://mlsanity.com/paper/2603.17227/) (2026)
- [LiveStre4m: Feed-Forward Live Streaming of Novel Views from Unposed Multi-View Video](https://mlsanity.com/paper/2604.06740/) (2026)
- [Stitch4D: Sparse Multi-Location 4D Urban Reconstruction via Spatio-Temporal Interpolation](https://mlsanity.com/paper/2604.07923/) (2026)
- [RobotPan: A 360^circ Surround-View Robotic Vision System for Embodied Perception](https://mlsanity.com/paper/2604.13476/) (2026)
- [DSD-GS: Dynamic-Static Decomposition of Gaussian Splatting for Efficient and High-Fidelity Dynamic Scene Reconstruction](https://mlsanity.com/paper/2605.30863/) (2026)
- [4D Reconstruction from Sparse Dynamic Cameras](https://mlsanity.com/paper/2606.04593/) (2026)
- [DLGStream: Dynamic Language-embedded Guassian Splatting for Open-vocabulary Enabled Free-viewpoint Video Streaming](https://mlsanity.com/paper/2606.28840/) (2026)
- [Online Neural Space Time Memory for Dynamic Novel View Synthesis](https://mlsanity.com/paper/2607.15271/) (2026)
- [S2GS: Structured Sparse Gaussian Streaming for Efficient Free-Viewpoint Video Reconstruction on Edge-IoT Devices](https://mlsanity.com/paper/2608.19639/) (2026)

## Abstract (arXiv)

Building Free-Viewpoint Videos in a streaming manner offers the advantage of rapid responsiveness compared to offline training methods, greatly enhancing user experience. However, current streaming approaches face challenges of high per-frame reconstruction time (10s+) and error accumulation, limiting their broader application. In this paper, we propose Instant Gaussian Stream (IGS), a fast and generalizable streaming framework, to address these issues. First, we introduce a generalized Anchor-driven Gaussian Motion Network, which projects multi-view 2D motion features into 3D space, using anchor points to drive the motion of all Gaussians. This generalized Network generates the motion of Gaussians for each target frame in the time required for a single inference. Second, we propose a Key-frame-guided Streaming Strategy that refines each key frame, enabling accurate reconstruction of temporally complex scenes while mitigating error accumulation. We conducted extensive in-domain and cross-domain evaluations, demonstrating that our approach can achieve streaming with a average per-frame reconstruction time of 2s+, alongside a enhancement in view synthesis quality.