# 3D Scene Rendering with Multimodal Gaussian Splatting

arXiv: https://arxiv.org/abs/2602.17124 · HTML: https://mlsanity.com/paper/2602.17124/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Builds on (most cited first, 9 of 9)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [K-Radar: 4D Radar Object Detection for Autonomous Driving in Various Weather Conditions](https://mlsanity.com/paper/2206.08171/) (2022)
- [3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities](https://mlsanity.com/paper/2407.17418/) (2024)
- [Surrogate modeling for Bayesian optimization beyond a single Gaussian process](https://mlsanity.com/paper/2205.14090/) (2022)
- [ActiveInitSplat: How Active Image Selection Helps Gaussian Splatting](https://mlsanity.com/paper/2503.06859/) (2025)
- [BOSfM: A View Planning Framework for Optimal 3D Reconstruction of Agricultural Scenes](https://mlsanity.com/paper/2509.24126/) (2025)

## Abstract (arXiv)

3D scene reconstruction and rendering are core tasks in computer vision, with applications spanning industrial monitoring, robotics, and autonomous driving. Recent advances in 3D Gaussian Splatting (GS) and its variants have achieved impressive rendering fidelity while maintaining high computational and memory efficiency. However, conventional vision-based GS pipelines typically rely on a sufficient number of camera views to initialize the Gaussian primitives and train their parameters, typically incurring additional processing cost during initialization while falling short in conditions where visual cues are unreliable, such as adverse weather, low illumination, or partial occlusions. To cope with these challenges, and motivated by the robustness of radio-frequency (RF) signals to weather, lighting, and occlusions, we introduce a multimodal framework that integrates RF sensing, such as automotive radar, with GS-based rendering as a more efficient and robust alternative to vision-only GS rendering. The proposed approach enables efficient depth prediction from only sparse RF-based depth measurements, yielding a high-quality 3D point cloud for initializing Gaussian functions across diverse GS architectures. Numerical tests demonstrate the merits of judiciously incorporating RF sensing into GS pipelines, achieving high-fidelity 3D scene rendering driven by RF-informed structural accuracy.