# RoutedFusion: Learning Real-time Depth Map Fusion

arXiv: https://arxiv.org/abs/2001.04388 · HTML: https://mlsanity.com/paper/2001.04388/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jan 2020.

## Built on by (most cited first, 30 of 52)

- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [Point-SLAM: Dense Neural Point Cloud-based SLAM](https://mlsanity.com/paper/2304.04278/) (2023)
- [Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting](https://mlsanity.com/paper/2312.10070/) (2023)
- [NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction](https://mlsanity.com/paper/2203.11283/) (2022)
- [Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians](https://mlsanity.com/paper/2405.16544/) (2024)
- [GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstruction](https://mlsanity.com/paper/2206.14735/) (2022)
- [Loopy-SLAM: Dense Neural SLAM with Loop Closures](https://mlsanity.com/paper/2402.09944/) (2024)
- [CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization](https://mlsanity.com/paper/2210.02757/) (2022)
- [VolumeFusion: Deep Depth Fusion for 3D Scene Reconstruction](https://mlsanity.com/paper/2108.08623/) (2021)
- [TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo](https://mlsanity.com/paper/2111.07418/) (2021)
- [GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM](https://mlsanity.com/paper/2403.19549/) (2024)
- [DiffComplete: Diffusion-based Generative 3D Shape Completion](https://mlsanity.com/paper/2306.16329/) (2023)
- [Joint Implicit Image Function for Guided Depth Super-Resolution](https://mlsanity.com/paper/2107.08717/) (2021)
- [Continual Neural Mapping: Learning An Implicit Scene Representation from Sequential Observations](https://mlsanity.com/paper/2108.05851/) (2021)
- [SurfelNeRF: Neural Surfel Radiance Fields for Online Photorealistic Reconstruction of Indoor Scenes](https://mlsanity.com/paper/2304.08971/) (2023)
- [MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction](https://mlsanity.com/paper/2308.08741/) (2023)
- [DNS SLAM: Dense Neural Semantic-Informed SLAM](https://mlsanity.com/paper/2312.00204/) (2023)
- [Marching-Primitives: Shape Abstraction from Signed Distance Function](https://mlsanity.com/paper/2303.13190/) (2023)
- [UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM](https://mlsanity.com/paper/2306.11048/) (2023)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [P2P-Bridge: Diffusion Bridges for 3D Point Cloud Denoising](https://mlsanity.com/paper/2408.16325/) (2024)
- [Probabilistic Volumetric Fusion for Dense Monocular SLAM](https://mlsanity.com/paper/2210.01276/) (2022)
- [CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstruction](https://mlsanity.com/paper/2304.14633/) (2023)
- [NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mapping](https://mlsanity.com/paper/2110.09415/) (2021)
- [Multi-sensor large-scale dataset for multi-view 3D reconstruction](https://mlsanity.com/paper/2203.06111/) (2022)
- [Tiled Multiplane Images for Practical 3D Photography](https://mlsanity.com/paper/2309.14291/) (2023)
- [A Real-Time Online Learning Framework for Joint 3D Reconstruction and Semantic Segmentation of Indoor Scenes](https://mlsanity.com/paper/2108.05246/) (2021)
- [Gradient-SDF: A Semi-Implicit Surface Representation for 3D Reconstruction](https://mlsanity.com/paper/2111.13652/) (2021)

## Abstract (arXiv)

The efficient fusion of depth maps is a key part of most state-of-the-art 3D reconstruction methods. Besides requiring high accuracy, these depth fusion methods need to be scalable and real-time capable. To this end, we present a novel real-time capable machine learning-based method for depth map fusion. Similar to the seminal depth map fusion approach by Curless and Levoy, we only update a local group of voxels to ensure real-time capability. Instead of a simple linear fusion of depth information, we propose a neural network that predicts non-linear updates to better account for typical fusion errors. Our network is composed of a 2D depth routing network and a 3D depth fusion network which efficiently handle sensor-specific noise and outliers. This is especially useful for surface edges and thin objects for which the original approach suffers from thickening artifacts. Our method outperforms the traditional fusion approach and related learned approaches on both synthetic and real data. We demonstrate the performance of our method in reconstructing fine geometric details from noise and outlier contaminated data on various scenes.