# Visual SLAM with 3D Gaussian Primitives and Depth Priors Enabling Novel View Synthesis

arXiv: https://arxiv.org/abs/2408.05635 · HTML: https://mlsanity.com/paper/2408.05635/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 10 Aug 2024.

## Builds on (most cited first, 21 of 21)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM](https://mlsanity.com/paper/2312.02126/) (2023)
- [Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis](https://mlsanity.com/paper/2312.16812/) (2023)
- [Approximate Differentiable Rendering with Algebraic Surfaces](https://mlsanity.com/paper/2207.10606/) (2022)
- [Learning the Distribution of Errors in Stereo Matching for Joint Disparity and Uncertainty Estimation](https://mlsanity.com/paper/2304.00152/) (2023)
- [Defending Graph Convolutional Networks against Dynamic Graph Perturbations via Bayesian Self-supervision](https://mlsanity.com/paper/2203.03762/) (2022)
- [Real-Time Dense 3D Mapping of Underwater Environments](https://mlsanity.com/paper/2304.02704/) (2023)
- [Benchmarking Neural Radiance Fields for Autonomous Robots: An Overview](https://mlsanity.com/paper/2405.05526/) (2024)
- [Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagation](https://mlsanity.com/paper/2208.09779/) (2022)
- [RED: A Systematic Real-Time Scheduling Approach for Robotic Environmental Dynamics](https://mlsanity.com/paper/2308.15368/) (2023)
- [Decentralized Adaptive Aerospace Transportation of Unknown Loads Using A Team of Robots](https://mlsanity.com/paper/2407.08084/) (2024)
- [Cocktail Edge Caching: Ride Dynamic Trends of Content Popularity with Ensemble Learning](https://mlsanity.com/paper/2101.05885/) (2021)
- [R^3: On-device Real-Time Deep Reinforcement Learning for Autonomous Robotics](https://mlsanity.com/paper/2308.15039/) (2023)
- [EDI: ESKF-based Disjoint Initialization for Visual-Inertial SLAM Systems](https://mlsanity.com/paper/2308.02670/) (2023)
- [BundledSLAM: An Accurate Visual SLAM System Using Multiple Cameras](https://mlsanity.com/paper/2403.19886/) (2024)
- [Predictive Edge Caching through Deep Mining of Sequential Patterns in User Content Retrievals](https://mlsanity.com/paper/2210.02657/) (2022)
- [Stereo-NEC: Enhancing Stereo Visual-Inertial SLAM Initialization with Normal Epipolar Constraints](https://mlsanity.com/paper/2403.07225/) (2024)
- [Adaptive Robot Detumbling of a Non-Rigid Satellite](https://mlsanity.com/paper/2407.17617/) (2024)
- [Robotic Inspection of Underground Utilities for Construction Survey Using a Ground Penetrating Radar](https://mlsanity.com/paper/2106.01907/) (2021)

## Built on by (most cited first, 2 of 2)

- [A Comprehensive Review on Noise Control of Diffusion Model](https://mlsanity.com/paper/2502.04669/) (2025)
- [Towards Next-Generation SLAM: A Survey on 3DGS-SLAM Focusing on Performance, Robustness, and Future Directions](https://mlsanity.com/paper/2602.04251/) (2026)

## Abstract (arXiv)

Conventional geometry-based SLAM systems lack dense 3D reconstruction capabilities since their data association usually relies on feature correspondences. Additionally, learning-based SLAM systems often fall short in terms of real-time performance and accuracy. Balancing real-time performance with dense 3D reconstruction capabilities is a challenging problem. In this paper, we propose a real-time RGB-D SLAM system that incorporates a novel view synthesis technique, 3D Gaussian Splatting, for 3D scene representation and pose estimation. This technique leverages the real-time rendering performance of 3D Gaussian Splatting with rasterization and allows for differentiable optimization in real time through CUDA implementation. We also enable mesh reconstruction from 3D Gaussians for explicit dense 3D reconstruction. To estimate accurate camera poses, we utilize a rotation-translation decoupled strategy with inverse optimization. This involves iteratively updating both in several iterations through gradient-based optimization. This process includes differentiably rendering RGB, depth, and silhouette maps and updating the camera parameters to minimize a combined loss of photometric loss, depth geometry loss, and visibility loss, given the existing 3D Gaussian map. However, 3D Gaussian Splatting (3DGS) struggles to accurately represent surfaces due to the multi-view inconsistency of 3D Gaussians, which can lead to reduced accuracy in both camera pose estimation and scene reconstruction. To address this, we utilize depth priors as additional regularization to enforce geometric constraints, thereby improving the accuracy of both pose estimation and 3D reconstruction. We also provide extensive experimental results on public benchmark datasets to demonstrate the effectiveness of our proposed methods in terms of pose accuracy, geometric accuracy, and rendering performance.