# MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements

arXiv: https://arxiv.org/abs/2404.00923 · HTML: https://mlsanity.com/paper/2404.00923/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 17 of 17)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM](https://mlsanity.com/paper/2312.02126/) (2023)
- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)
- [GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting](https://mlsanity.com/paper/2311.11700/) (2023)
- [COLMAP-Free 3D Gaussian Splatting](https://mlsanity.com/paper/2312.07504/) (2023)
- [Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting](https://mlsanity.com/paper/2312.10070/) (2023)
- [BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration](https://mlsanity.com/paper/1604.01093/) (2016)
- [The TUM VI Benchmark for Evaluating Visual-Inertial Odometry](https://mlsanity.com/paper/1804.06120/) (2018)
- [Comp4D: LLM-Guided Compositional 4D Scene Generation](https://mlsanity.com/paper/2403.16993/) (2024)
- [LIV-GaussMap: LiDAR-Inertial-Visual Fusion for Real-time 3D Radiance Field Map Rendering](https://mlsanity.com/paper/2401.14857/) (2024)

## Built on by (most cited first, 22 of 22)

- [Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision](https://mlsanity.com/paper/2504.02477/) (2025)
- [On The Planning Abilities of OpenAI's o1 Models: Feasibility, Optimality, and Generalizability](https://mlsanity.com/paper/2409.19924/) (2024)
- [Advancing Extended Reality with 3D Gaussian Splatting: Innovations and Prospects](https://mlsanity.com/paper/2412.06257/) (2024)
- [LongSplat: Online Generalizable 3D Gaussian Splatting from Long Sequence Images](https://mlsanity.com/paper/2507.16144/) (2025)
- [DynaGSLAM: Real-Time Gaussian-Splatting SLAM for Online Rendering, Tracking, Motion Predictions of Moving Objects in Dynamic Scenes](https://mlsanity.com/paper/2503.11979/) (2025)
- [GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multi-sensor Fused Odometry with Gaussian Mapping](https://mlsanity.com/paper/2501.08672/) (2025)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [VIGS-SLAM: Visual Inertial Gaussian Splatting SLAM](https://mlsanity.com/paper/2512.02293/) (2025)
- [FGS-SLAM: Fourier-based Gaussian Splatting for Real-time SLAM with Sparse and Dense Map Fusion](https://mlsanity.com/paper/2503.01109/) (2025)
- [Robust LiDAR-Camera Calibration with 2D Gaussian Splatting](https://mlsanity.com/paper/2504.00525/) (2025)
- [Towards Next-Generation SLAM: A Survey on 3DGS-SLAM Focusing on Performance, Robustness, and Future Directions](https://mlsanity.com/paper/2602.04251/) (2026)
- [Learning Fine-Grained Geometry for Sparse-View Splatting via Cascade Depth Loss](https://mlsanity.com/paper/2505.22279/) (2025)
- [MrGS: Multi-modal Radiance Fields with 3D Gaussian Splatting for RGB-Thermal Novel View Synthesis](https://mlsanity.com/paper/2511.22997/) (2025)
- [Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis](https://mlsanity.com/paper/2607.22147/) (2026)
- [UDGS-SLAM : UniDepth Assisted Gaussian Splatting for Monocular SLAM](https://mlsanity.com/paper/2409.00362/) (2024)
- [Drive-Through 3D Vehicle Exterior Reconstruction via Dynamic-Scene SfM and Distortion-Aware Gaussian Splatting](https://mlsanity.com/paper/2603.26638/) (2026)
- [GaussianFlow SLAM: Monocular Gaussian Splatting SLAM Guided by GaussianFlow](https://mlsanity.com/paper/2604.15612/) (2026)
- [MAGiSt3R: Multi-Agent Feed-forward 3D Reconstruction from Monocular RGB Videos](https://mlsanity.com/paper/2607.15211/) (2026)
- [Stipple: Real-Time Incremental Gaussian Splatting with Visual-Inertial Tracking](https://mlsanity.com/paper/2608.00931/) (2026)
- [CGS-SLAM: Collaborative Gaussian Splatting based SLAM for Multi-Agent Reconstruction](https://mlsanity.com/paper/2608.26868/) (2026)
- [Cube-Splat: High-Fidelity 360deg Gaussian Splatting SLAM via Cubemap Factorization and Adjoint-Consistent Optimization](https://mlsanity.com/paper/2609.21347/) (2026)
- [LiTe-GS: Oracle-Efficient Next Best View Selection for 3D Gaussian Splatting](https://mlsanity.com/paper/2609.30393/) (2026)

## Abstract (arXiv)

Simultaneous localization and mapping is essential for position tracking and scene understanding. 3D Gaussian-based map representations enable photorealistic reconstruction and real-time rendering of scenes using multiple posed cameras. We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM. Our method, MM3DGS, addresses the limitations of prior neural radiance field-based representations by enabling faster rendering, scale awareness, and improved trajectory tracking. Our framework enables keyframe-based mapping and tracking utilizing loss functions that incorporate relative pose transformations from pre-integrated inertial measurements, depth estimates, and measures of photometric rendering quality. We also release a multi-modal dataset, UT-MM, collected from a mobile robot equipped with a camera and an inertial measurement unit. Experimental evaluation on several scenes from the dataset shows that MM3DGS achieves 3x improvement in tracking and 5% improvement in photometric rendering quality compared to the current 3DGS SLAM state-of-the-art, while allowing real-time rendering of a high-resolution dense 3D map. Project Webpage: https://vita-group.github.io/MM3DGS-SLAM