# AONeuS: A Neural Rendering Framework for Acoustic-Optical Sensor Fusion

arXiv: https://arxiv.org/abs/2402.03309 · HTML: https://mlsanity.com/paper/2402.03309/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 68 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis](https://mlsanity.com/paper/2302.14859/) (2023)
- [SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2206.05737/) (2022)
- [Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields](https://mlsanity.com/paper/2309.03185/) (2023)
- [T"oRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis](https://mlsanity.com/paper/2109.15271/) (2021)
- [SeaThru-NeRF: Neural Radiance Fields in Scattering Media](https://mlsanity.com/paper/2304.07743/) (2023)
- [FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information](https://mlsanity.com/paper/2311.17874/) (2023)
- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [WaterNeRF: Neural Radiance Fields for Underwater Scenes](https://mlsanity.com/paper/2209.13091/) (2022)
- [ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering](https://mlsanity.com/paper/2305.02103/) (2023)
- [Cross-Spectral Neural Radiance Fields](https://mlsanity.com/paper/2209.00648/) (2022)
- [Neural Implicit Surface Reconstruction using Imaging Sonar](https://mlsanity.com/paper/2209.08221/) (2022)
- [Transient Neural Radiance Fields for Lidar View Synthesis and 3D Reconstruction](https://mlsanity.com/paper/2307.09555/) (2023)
- [Neural Volumetric Reconstruction for Coherent Synthetic Aperture Sonar](https://mlsanity.com/paper/2306.09909/) (2023)
- [Neural Spline Fields for Burst Image Fusion and Layer Separation](https://mlsanity.com/paper/2312.14235/) (2023)
- [The Implicit Values of A Good Hand Shake: Handheld Multi-Frame Neural Depth Refinement](https://mlsanity.com/paper/2111.13738/) (2021)
- [CLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations](https://mlsanity.com/paper/2209.01194/) (2022)
- [DehazeNeRF: Multiple Image Haze Removal and 3D Shape Reconstruction using Neural Radiance Fields](https://mlsanity.com/paper/2303.11364/) (2023)
- [Neural Spectro-polarimetric Fields](https://mlsanity.com/paper/2306.12562/) (2023)
- [Spatial Acoustic Projection for 3D Imaging Sonar Reconstruction](https://mlsanity.com/paper/2206.02840/) (2022)
- [Conditional GANs for Sonar Image Filtering with Applications to Underwater Occupancy Mapping](https://mlsanity.com/paper/2209.11757/) (2022)

## Built on by (most cited first, 13 of 13)

- [Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion](https://mlsanity.com/paper/2404.04687/) (2024)
- [Blurred LiDAR for Sharper 3D: Robust Handheld 3D Scanning with Diffuse LiDAR and RGB](https://mlsanity.com/paper/2411.19474/) (2024)
- [UWStereo: A Large Synthetic Dataset for Underwater Stereo Matching](https://mlsanity.com/paper/2409.01782/) (2024)
- [Transientangelo: Few-Viewpoint Surface Reconstruction Using Single-Photon Lidar](https://mlsanity.com/paper/2408.12191/) (2024)
- [SonarSplat: Novel View Synthesis of Imaging Sonar via Gaussian Splatting](https://mlsanity.com/paper/2504.00159/) (2025)
- [Acoustic Neural 3D Reconstruction Under Pose Drift](https://mlsanity.com/paper/2503.08930/) (2025)
- [SH-SAS: An Implicit Neural Representation for Complex Spherical-Harmonic Scattering Fields for 3D Synthetic Aperture Sonar](https://mlsanity.com/paper/2509.11087/) (2025)
- [SonarSweep: Fusing Sonar and Vision for Robust 3D Reconstruction via Plane Sweeping](https://mlsanity.com/paper/2511.00392/) (2025)
- [NAS-GS: Noise-Aware Sonar Gaussian Splatting](https://mlsanity.com/paper/2601.06285/) (2026)
- [TouchAnything: Diffusion-Guided 3D Reconstruction from Sparse Robot Touches](https://mlsanity.com/paper/2604.08945/) (2026)
- [Single View Seafloor Recovery from Imaging Sonar via Differentiable Rendering](https://mlsanity.com/paper/2605.24195/) (2026)
- [One Model, Two Worlds: Bidirectional Sonar-Optical Translation](https://mlsanity.com/paper/2609.06253/) (2026)
- [Adapting Vision Foundation Models to Acoustics for Pose-Free 3D Sonar Reconstruction](https://mlsanity.com/paper/2609.06261/) (2026)

## Abstract (arXiv)

Underwater perception and 3D surface reconstruction are challenging problems with broad applications in construction, security, marine archaeology, and environmental monitoring. Treacherous operating conditions, fragile surroundings, and limited navigation control often dictate that submersibles restrict their range of motion and, thus, the baseline over which they can capture measurements. In the context of 3D scene reconstruction, it is well-known that smaller baselines make reconstruction more challenging. Our work develops a physics-based multimodal acoustic-optical neural surface reconstruction framework (AONeuS) capable of effectively integrating high-resolution RGB measurements with low-resolution depth-resolved imaging sonar measurements. By fusing these complementary modalities, our framework can reconstruct accurate high-resolution 3D surfaces from measurements captured over heavily-restricted baselines. Through extensive simulations and in-lab experiments, we demonstrate that AONeuS dramatically outperforms recent RGB-only and sonar-only inverse-differentiable-rendering--based surface reconstruction methods. A website visualizing the results of our paper is located at this address: https://aoneus.github.io/