# Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion

arXiv: https://arxiv.org/abs/2404.04687 · HTML: https://mlsanity.com/paper/2404.04687/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 6 Apr 2024.

## Builds on (most cited first, 27 of 27)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting](https://mlsanity.com/paper/2310.10642/) (2023)
- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM](https://mlsanity.com/paper/2312.02126/) (2023)
- [SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes](https://mlsanity.com/paper/2312.14937/) (2023)
- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)
- [GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting](https://mlsanity.com/paper/2311.11700/) (2023)
- [GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces](https://mlsanity.com/paper/2311.17977/) (2023)
- [Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images](https://mlsanity.com/paper/2311.13398/) (2023)
- [GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions](https://mlsanity.com/paper/2311.16037/) (2023)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023)
- [DynMF: Neural Motion Factorization for Real-time Dynamic View Synthesis with 3D Gaussian Splatting](https://mlsanity.com/paper/2312.00112/) (2023)
- [3D Gaussian as a New Era: A Survey](https://mlsanity.com/paper/2402.07181/) (2024)
- [T"oRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis](https://mlsanity.com/paper/2109.15271/) (2021)
- [Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction](https://mlsanity.com/paper/2312.01196/) (2023)
- [CoGS: Controllable Gaussian Splatting](https://mlsanity.com/paper/2312.05664/) (2023)
- [Point'n Move: Interactive Scene Object Manipulation on Gaussian Splatting Radiance Fields](https://mlsanity.com/paper/2311.16737/) (2023)
- [Neural Implicit Surface Reconstruction using Imaging Sonar](https://mlsanity.com/paper/2209.08221/) (2022)
- [Neural Volumetric Reconstruction for Coherent Synthetic Aperture Sonar](https://mlsanity.com/paper/2306.09909/) (2023)
- [High-Fidelity SLAM Using Gaussian Splatting with Rendering-Guided Densification and Regularized Optimization](https://mlsanity.com/paper/2403.12535/) (2024)
- [AONeuS: A Neural Rendering Framework for Acoustic-Optical Sensor Fusion](https://mlsanity.com/paper/2402.03309/) (2024)
- [SINR: Deconvolving Circular SAS Images Using Implicit Neural Representations](https://mlsanity.com/paper/2204.10428/) (2022)
- [Conditional GANs for Sonar Image Filtering with Applications to Underwater Occupancy Mapping](https://mlsanity.com/paper/2209.11757/) (2022)
- [Monocular Camera and Single-Beam Sonar-Based Underwater Collision-Free Navigation with Domain Randomization](https://mlsanity.com/paper/2212.04373/) (2022)

## Built on by (most cited first, 17 of 17)

- [RadarSplat: Radar Gaussian Splatting for High-Fidelity Data Synthesis and 3D Reconstruction of Autonomous Driving Scenes](https://mlsanity.com/paper/2506.01379/) (2025)
- [Volumetrically Consistent 3D Gaussian Rasterization](https://mlsanity.com/paper/2412.03378/) (2024)
- [UWStereo: A Large Synthetic Dataset for Underwater Stereo Matching](https://mlsanity.com/paper/2409.01782/) (2024)
- [Gaussian Splashing: Direct Volumetric Rendering Underwater](https://mlsanity.com/paper/2411.19588/) (2024)
- [SonarSplat: Novel View Synthesis of Imaging Sonar via Gaussian Splatting](https://mlsanity.com/paper/2504.00159/) (2025)
- [CompGS++: Compressed Gaussian Splatting for Static and Dynamic Scene Representation](https://mlsanity.com/paper/2504.13022/) (2025)
- [Acoustic Neural 3D Reconstruction Under Pose Drift](https://mlsanity.com/paper/2503.08930/) (2025)
- [Time of the Flight of the Gaussians: Optimizing Depth Indirectly in Dynamic Radiance Fields](https://mlsanity.com/paper/2505.05356/) (2025)
- [SonarSweep: Fusing Sonar and Vision for Robust 3D Reconstruction via Plane Sweeping](https://mlsanity.com/paper/2511.00392/) (2025)
- [NAS-GS: Noise-Aware Sonar Gaussian Splatting](https://mlsanity.com/paper/2601.06285/) (2026)
- [TouchAnything: Diffusion-Guided 3D Reconstruction from Sparse Robot Touches](https://mlsanity.com/paper/2604.08945/) (2026)
- [On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting](https://mlsanity.com/paper/2607.08250/) (2026)
- [Underwater Monocular Metric Depth Estimation: Real-World Benchmarks and Synthetic Fine-Tuning with Vision Foundation Models](https://mlsanity.com/paper/2507.02148/) (2025)
- [Non-line-of-sight imaging with arbitrary relay surface geometries via 3D Gaussian Transient Rendering](https://mlsanity.com/paper/2606.21270/) (2026)
- [Adapting Vision Foundation Models to Acoustics for Pose-Free 3D Sonar Reconstruction](https://mlsanity.com/paper/2609.06261/) (2026)
- [3D Point Splatting for mmWave Radar Novel View Synthesis](https://mlsanity.com/paper/2609.11894/) (2026)
- [Mobile-4DGS: Unified Static-Dynamic Real-time Mobile Gaussian Splatting](https://mlsanity.com/paper/2610.05289/) (2026)

## Abstract (arXiv)

Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent technique in computer vision and graphics for reconstructing 3D scenes. GS represents a scene as a set of 3D Gaussians with varying opacities and employs a computationally efficient splatting operation along with analytical derivatives to compute the 3D Gaussian parameters given scene images captured from various viewpoints. Unfortunately, capturing surround view (360circ viewpoint) images is impossible or impractical in many real-world imaging scenarios, including underwater imaging, rooms inside a building, and autonomous navigation. In these restricted baseline imaging scenarios, the GS algorithm suffers from a well-known 'missing cone' problem, which results in poor reconstruction along the depth axis. In this manuscript, we demonstrate that using transient data (from sonars) allows us to address the missing cone problem by sampling high-frequency data along the depth axis. We extend the Gaussian splatting algorithms for two commonly used sonars and propose fusion algorithms that simultaneously utilize RGB camera data and sonar data. Through simulations, emulations, and hardware experiments across various imaging scenarios, we show that the proposed fusion algorithms lead to significantly better novel view synthesis (5 dB improvement in PSNR) and 3D geometry reconstruction (60% lower Chamfer distance).