# SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction

arXiv: https://arxiv.org/abs/2409.11211 · HTML: https://mlsanity.com/paper/2409.11211/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 72 of its 93 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 30 of 72)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering](https://mlsanity.com/paper/2311.12775/) (2023)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting](https://mlsanity.com/paper/2310.10642/) (2023)
- [Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis](https://mlsanity.com/paper/2104.00677/) (2021)
- [LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS](https://mlsanity.com/paper/2311.17245/) (2023)

## Built on by (most cited first, 30 of 52)

- [SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes](https://mlsanity.com/paper/2410.17249/) (2024)
- [Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Video](https://mlsanity.com/paper/2412.06424/) (2024)
- [SplatFormer: Point Transformer for Robust 3D Gaussian Splatting](https://mlsanity.com/paper/2411.06390/) (2024)
- [Tackling View-Dependent Semantics in 3D Language Gaussian Splatting](https://mlsanity.com/paper/2505.24746/) (2025)
- [NeRF Is a Valuable Assistant for 3D Gaussian Splatting](https://mlsanity.com/paper/2507.23374/) (2025)
- [Feature4X: Bridging Any Monocular Video to 4D Agentic AI with Versatile Gaussian Feature Fields](https://mlsanity.com/paper/2503.20776/) (2025)
- [Hybrid 3D-4D Gaussian Splatting for Fast Dynamic Scene Representation](https://mlsanity.com/paper/2505.13215/) (2025)
- [FreeGave: 3D Physics Learning from Dynamic Videos by Gaussian Velocity](https://mlsanity.com/paper/2506.07865/) (2025)
- [Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming](https://mlsanity.com/paper/2503.08166/) (2025)
- [AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2601.00796/) (2026)
- [Splat the Net: Radiance Fields with Splattable Neural Primitives](https://mlsanity.com/paper/2510.08491/) (2025)
- [GASPACHO: Gaussian Splatting for Controllable Humans and Objects](https://mlsanity.com/paper/2503.09342/) (2025)
- [Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splatting](https://mlsanity.com/paper/2503.14698/) (2025)
- [Dropping Anchor and Spherical Harmonics for Sparse-view Gaussian Splatting](https://mlsanity.com/paper/2602.20933/) (2026)
- [MGSR: 2D/3D Mutual-boosted Gaussian Splatting for High-fidelity Surface Reconstruction under Various Light Conditions](https://mlsanity.com/paper/2503.05182/) (2025)
- [ContraGS: Codebook-Condensed and Trainable Gaussian Splatting for Fast, Memory-Efficient Reconstruction](https://mlsanity.com/paper/2509.03775/) (2025)
- [AHA! Animating Human Avatars in Diverse Scenes with Gaussian Splatting](https://mlsanity.com/paper/2511.09827/) (2025)
- [GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction](https://mlsanity.com/paper/2512.25073/) (2025)
- [RelayGS: Reconstructing Dynamic Scenes with Large-Scale and Complex Motions via Relay Gaussians](https://mlsanity.com/paper/2412.02493/) (2024)
- [Spline Deformation Field](https://mlsanity.com/paper/2507.07521/) (2025)
- [ECoNGS: Efficient Compressive Neural Gaussian Splats for Volume Visualization](https://mlsanity.com/paper/2607.18466/) (2026)
- [AI-Driven Innovations in Volumetric Video Streaming: A Review](https://mlsanity.com/paper/2412.12208/) (2024)
- [SmallGS: Gaussian Splatting-based Camera Pose Estimation for Small-Baseline Videos](https://mlsanity.com/paper/2504.17810/) (2025)
- [BRUM: Robust 3D Vehicle Reconstruction from 360 Sparse Images](https://mlsanity.com/paper/2507.12095/) (2025)
- [Splannequin: Freezing Monocular Mannequin-Challenge Footage with Dual-Detection Splatting](https://mlsanity.com/paper/2512.05113/) (2025)
- [Joint Semantic and Rendering Enhancements in 3D Gaussian Modeling with Anisotropic Local Encoding](https://mlsanity.com/paper/2601.02339/) (2026)
- [Ego-1K -- A Large-Scale Multiview Video Dataset for Egocentric Vision](https://mlsanity.com/paper/2603.13741/) (2026)
- [AHOY! Animatable Humans under Occlusion from YouTube Videos with Gaussian Splatting and Video Diffusion Priors](https://mlsanity.com/paper/2603.17975/) (2026)
- [DOC-GS: Dual-Domain Observation and Calibration for Reliable Sparse-View Gaussian Splatting](https://mlsanity.com/paper/2604.06739/) (2026)
- [Reconstruction Using the Invisible: Intuition from NIR and Metadata for Enhanced 3D Gaussian Splatting](https://mlsanity.com/paper/2508.14443/) (2025)

## Abstract (arXiv)

Digitizing 3D static scenes and 4D dynamic events from multi-view images has long been a challenge in computer vision and graphics. Recently, 3D Gaussian Splatting (3DGS) has emerged as a practical and scalable reconstruction method, gaining popularity due to its impressive reconstruction quality, real-time rendering capabilities, and compatibility with widely used visualization tools. However, the method requires a substantial number of input views to achieve high-quality scene reconstruction, introducing a significant practical bottleneck. This challenge is especially severe in capturing dynamic scenes, where deploying an extensive camera array can be prohibitively costly. In this work, we identify the lack of spatial autocorrelation of splat features as one of the factors contributing to the suboptimal performance of the 3DGS technique in sparse reconstruction settings. To address the issue, we propose an optimization strategy that effectively regularizes splat features by modeling them as the outputs of a corresponding implicit neural field. This results in a consistent enhancement of reconstruction quality across various scenarios. Our approach effectively handles static and dynamic cases, as demonstrated by extensive testing across different setups and scene complexities.