# Real-Time Dense Mapping for Self-driving Vehicles using Fisheye Cameras

arXiv: https://arxiv.org/abs/1809.06132 · HTML: https://mlsanity.com/paper/1809.06132/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2018.

## Built on by (most cited first, 10 of 10)

- [RigNet: Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2107.13802/) (2021)
- [Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges](https://mlsanity.com/paper/2205.13281/) (2022)
- [SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion](https://mlsanity.com/paper/2204.13635/) (2022)
- [RigNet++: Semantic Assisted Repetitive Image Guided Network for Depth Completion](https://mlsanity.com/paper/2309.00655/) (2023)
- [Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction](https://mlsanity.com/paper/2502.09563/) (2025)
- [Depth360: Self-supervised Learning for Monocular Depth Estimation using Learnable Camera Distortion Model](https://mlsanity.com/paper/2110.10415/) (2021)
- [ROSEFusion: Random Optimization for Online Dense Reconstruction under Fast Camera Motion](https://mlsanity.com/paper/2105.05600/) (2021)
- [LightDepth: A Resource Efficient Depth Estimation Approach for Dealing with Ground Truth Sparsity via Curriculum Learning](https://mlsanity.com/paper/2211.08608/) (2022)
- [A Novel Viewport-Adaptive Motion Compensation Technique for Fisheye Video](https://mlsanity.com/paper/2202.13892/) (2022)
- [Reference-Free Omnidirectional Stereo Matching via Multi-View Consistency Maximization](https://mlsanity.com/paper/2603.15019/) (2026)

## Abstract (arXiv)

We present a real-time dense geometric mapping algorithm for large-scale environments. Unlike existing methods which use pinhole cameras, our implementation is based on fisheye cameras which have larger field of view and benefit some other tasks including Visual-Inertial Odometry, localization and object detection around vehicles. Our algorithm runs on in-vehicle PCs at 15 Hz approximately, enabling vision-only 3D scene perception for self-driving vehicles. For each synchronized set of images captured by multiple cameras, we first compute a depth map for a reference camera using plane-sweeping stereo. To maintain both accuracy and efficiency, while accounting for the fact that fisheye images have a rather low resolution, we recover the depths using multiple image resolutions. We adopt the fast object detection framework YOLOv3 to remove potentially dynamic objects. At the end of the pipeline, we fuse the fisheye depth images into the truncated signed distance function (TSDF) volume to obtain a 3D map. We evaluate our method on large-scale urban datasets, and results show that our method works well even in complex environments.