# SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection

arXiv: https://arxiv.org/abs/2601.03024 · HTML: https://mlsanity.com/paper/2601.03024/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 68 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Builds on (most cited first, 30 of 47)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization](https://mlsanity.com/paper/2403.06912/) (2024)
- [Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis](https://mlsanity.com/paper/2401.02436/) (2023)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [WildGaussians: 3D Gaussian Splatting in the Wild](https://mlsanity.com/paper/2407.08447/) (2024)
- [ActiveNeRF: Learning where to See with Uncertainty Estimation](https://mlsanity.com/paper/2209.08546/) (2022)
- [PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting](https://mlsanity.com/paper/2406.10219/) (2024)
- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [Hard negative examples are hard, but useful](https://mlsanity.com/paper/2007.12749/) (2020)
- [Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations](https://mlsanity.com/paper/2109.02123/) (2021)
- [Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification](https://mlsanity.com/paper/2203.10192/) (2022)
- [DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting](https://mlsanity.com/paper/2504.00773/) (2025)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fields](https://mlsanity.com/paper/2209.08718/) (2022)
- [Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration](https://mlsanity.com/paper/2502.16652/) (2025)
- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [MVPGS: Excavating Multi-view Priors for Gaussian Splatting from Sparse Input Views](https://mlsanity.com/paper/2409.14316/) (2024)
- [NARUTO: Neural Active Reconstruction from Uncertain Target Observations](https://mlsanity.com/paper/2402.18771/) (2024)
- [MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion](https://mlsanity.com/paper/2504.20040/) (2025)
- [MACARONS: Mapping And Coverage Anticipation with RGB Online Self-Supervision](https://mlsanity.com/paper/2303.03315/) (2023)
- [Improving NeRF Quality by Progressive Camera Placement for Unrestricted Navigation in Complex Environments](https://mlsanity.com/paper/2309.00014/) (2023)
- [ActiveGAMER: Active GAussian Mapping through Efficient Rendering](https://mlsanity.com/paper/2501.06897/) (2025)

## Built on by (most cited first, 2 of 2)

- [2D GauSS-MI: Efficient Active Scene Reconstruction with Balanced Visual and Geometric Quality](https://mlsanity.com/paper/2609.21516/) (2026)
- [Matisse: Evidence-Space Reasoning for Active 3D Reconstruction](https://mlsanity.com/paper/2609.38746/) (2026)

## Abstract (arXiv)

We propose Self-Augmented Residual 3D Gaussian Splatting (SA-ResGS), a novel framework to stabilize uncertainty quantification and enhancing uncertainty-aware supervision in next-best-view (NBV) selection for active scene reconstruction. SA-ResGS improves both the reliability of uncertainty estimates and their effectiveness for supervision by generating Self-Augmented point clouds (SA-Points) via triangulation between a training view and a rasterized extrapolated view, enabling efficient scene coverage estimation. While improving scene coverage through physically guided view selection, SA-ResGS also addresses the challenge of under-supervised Gaussians, exacerbated by sparse and wide-baseline views, by introducing the first residual learning strategy tailored for 3D Gaussian Splatting. This targeted supervision enhances gradient flow in high-uncertainty Gaussians by combining uncertainty-driven filtering with dropout- and hard-negative-mining-inspired sampling. Our contributions are threefold: (1) a physically grounded view selection strategy that promotes efficient and uniform scene coverage; (2) an uncertainty-aware residual supervision scheme that amplifies learning signals for weakly contributing Gaussians, improving training stability and uncertainty estimation across scenes with diverse camera distributions; (3) an implicit unbiasing of uncertainty quantification as a consequence of constrained view selection and residual supervision, which together mitigate conflicting effects of wide-baseline exploration and sparse-view ambiguity in NBV planning. Experiments on active view selection demonstrate that SA-ResGS outperforms state-of-the-art baselines in both reconstruction quality and view selection robustness.