# Modeling Uncertainty in 3D Gaussian Splatting through Continuous Semantic Splatting

arXiv: https://arxiv.org/abs/2411.02547 · HTML: https://mlsanity.com/paper/2411.02547/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 4 Nov 2024.

## Builds on (most cited first, 25 of 25)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [LangSplat: 3D Language Gaussian Splatting](https://mlsanity.com/paper/2312.16084/) (2023)
- [Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields](https://mlsanity.com/paper/2312.03203/) (2023)
- [MP3: A Unified Model to Map, Perceive, Predict and Plan](https://mlsanity.com/paper/2101.06806/) (2021)
- [Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow](https://mlsanity.com/paper/1802.07095/) (2018)
- [3D Gaussian as a New Era: A Survey](https://mlsanity.com/paper/2402.07181/) (2024)
- [PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting](https://mlsanity.com/paper/2406.10219/) (2024)
- [Learning Navigation Behaviors End-to-End with AutoRL](https://mlsanity.com/paper/1809.10124/) (2018)
- [A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers](https://mlsanity.com/paper/2207.00254/) (2022)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [LoopSplat: Loop Closure by Registering 3D Gaussian Splats](https://mlsanity.com/paper/2408.10154/) (2024)
- [Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mapping](https://mlsanity.com/paper/1909.04631/) (2019)
- [Modeling uncertainty for Gaussian Splatting](https://mlsanity.com/paper/2403.18476/) (2024)
- [Semantic 3D Occupancy Mapping through Efficient High Order CRFs](https://mlsanity.com/paper/1707.07388/) (2017)
- [These Maps Are Made For Walking: Real-Time Terrain Property Estimation for Mobile Robots](https://mlsanity.com/paper/2205.12925/) (2022)
- [ConvBKI: Real-Time Probabilistic Semantic Mapping Network with Quantifiable Uncertainty](https://mlsanity.com/paper/2310.16020/) (2023)
- [Convolutional Bayesian Kernel Inference for 3D Semantic Mapping](https://mlsanity.com/paper/2209.10663/) (2022)
- [Gaussian Processes Semantic Map Representation](https://mlsanity.com/paper/1707.01532/) (2017)
- [Evidential Semantic Mapping in Off-road Environments with Uncertainty-aware Bayesian Kernel Inference](https://mlsanity.com/paper/2403.14138/) (2024)

## Built on by (most cited first, 12 of 12)

- [POp-GS: Next Best View in 3D-Gaussian Splatting with P-Optimality](https://mlsanity.com/paper/2503.07819/) (2025)
- [These Magic Moments: Differentiable Uncertainty Quantification of Radiance Field Models](https://mlsanity.com/paper/2503.14665/) (2025)
- [PRIMU: Uncertainty Estimation for Novel Views in Gaussian Splatting from Primitive-Based Representations of Error and Coverage](https://mlsanity.com/paper/2508.02443/) (2025)
- [Bayesian Fields: Task-driven Open-Set Semantic Gaussian Splatting](https://mlsanity.com/paper/2503.05949/) (2025)
- [View-Dependent Uncertainty Estimation of 3D Gaussian Splatting](https://mlsanity.com/paper/2504.07370/) (2025)
- [Understanding while Exploring: Semantics-driven Active Mapping](https://mlsanity.com/paper/2506.00225/) (2025)
- [Predictive Photometric Uncertainty in Gaussian Splatting for Novel View Synthesis](https://mlsanity.com/paper/2603.22786/) (2026)
- [Uncertainty-driven 3D Gaussian Splatting Active Mapping via Anisotropic Visibility Field](https://mlsanity.com/paper/2605.30342/) (2026)
- [VarSplat: Uncertainty-aware 3D Gaussian Splatting for Robust RGB-D SLAM](https://mlsanity.com/paper/2603.09673/) (2026)
- [Next Best View Selections for Semantic and Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2512.22771/) (2025)
- [ConfCtrl: Enabling Precise Camera Control in Video Diffusion via Confidence-Aware Interpolation](https://mlsanity.com/paper/2603.09819/) (2026)
- [LiTe-GS: Oracle-Efficient Next Best View Selection for 3D Gaussian Splatting](https://mlsanity.com/paper/2609.30393/) (2026)

## Abstract (arXiv)

In this paper, we present a novel algorithm for probabilistically updating and rasterizing semantic maps within 3D Gaussian Splatting (3D-GS). Although previous methods have introduced algorithms which learn to rasterize features in 3D-GS for enhanced scene understanding, 3D-GS can fail without warning which presents a challenge for safety-critical robotic applications. To address this gap, we propose a method which advances the literature of continuous semantic mapping from voxels to ellipsoids, combining the precise structure of 3D-GS with the ability to quantify uncertainty of probabilistic robotic maps. Given a set of images, our algorithm performs a probabilistic semantic update directly on the 3D ellipsoids to obtain an expectation and variance through the use of conjugate priors. We also propose a probabilistic rasterization which returns per-pixel segmentation predictions with quantifiable uncertainty. We compare our method with similar probabilistic voxel-based methods to verify our extension to 3D ellipsoids, and perform ablation studies on uncertainty quantification and temporal smoothing.