# Full Surround Monodepth from Multiple Cameras

arXiv: https://arxiv.org/abs/2104.00152 · HTML: https://mlsanity.com/paper/2104.00152/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 54 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Learning a Multi-View Stereo Machine](https://mlsanity.com/paper/1708.05375/) (2017)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras](https://mlsanity.com/paper/1904.04998/) (2019)
- [Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction](https://mlsanity.com/paper/1803.03893/) (2018)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance](https://mlsanity.com/paper/2007.06936/) (2020)
- [DPSNet: End-to-end Deep Plane Sweep Stereo](https://mlsanity.com/paper/1905.00538/) (2019)
- [Semi-Supervised Deep Learning for Monocular Depth Map Prediction](https://mlsanity.com/paper/1702.02706/) (2017)
- [UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning](https://mlsanity.com/paper/1709.06841/) (2017)
- [SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation](https://mlsanity.com/paper/1810.01849/) (2018)
- [MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images](https://mlsanity.com/paper/2008.06534/) (2020)
- [Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency](https://mlsanity.com/paper/1905.02706/) (2019)
- [360SD-Net: 360deg Stereo Depth Estimation with Learnable Cost Volume](https://mlsanity.com/paper/1911.04460/) (2019)

## Built on by (most cited first, 30 of 41)

- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction](https://mlsanity.com/paper/2311.12754/) (2023)
- [SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2204.03636/) (2022)
- [Robust Monocular Depth Estimation under Challenging Conditions](https://mlsanity.com/paper/2308.09711/) (2023)
- [DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Input](https://mlsanity.com/paper/2409.12753/) (2024)
- [OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments](https://mlsanity.com/paper/2312.09243/) (2023)
- [R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras](https://mlsanity.com/paper/2308.14713/) (2023)
- [TiG-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learning](https://mlsanity.com/paper/2212.13979/) (2022)
- [OccFusion: Multi-Sensor Fusion Framework for 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2403.01644/) (2024)
- [Learning Optical Flow, Depth, and Scene Flow without Real-World Labels](https://mlsanity.com/paper/2203.15089/) (2022)
- [RSA: Resolving Scale Ambiguities in Monocular Depth Estimators through Language Descriptions](https://mlsanity.com/paper/2410.02924/) (2024)
- [Multi-Camera Collaborative Depth Prediction via Consistent Structure Estimation](https://mlsanity.com/paper/2210.02009/) (2022)
- [EGA-Depth: Efficient Guided Attention for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2304.03369/) (2023)
- [InverseMatrixVT3D: An Efficient Projection Matrix-Based Approach for 3D Occupancy Prediction](https://mlsanity.com/paper/2401.12422/) (2024)
- [HybridOcc: NeRF Enhanced Transformer-based Multi-Camera 3D Occupancy Prediction](https://mlsanity.com/paper/2408.09104/) (2024)
- [SfM-TTR: Using Structure from Motion for Test-Time Refinement of Single-View Depth Networks](https://mlsanity.com/paper/2211.13551/) (2022)
- [M2Depth: Self-supervised Two-Frame Multi-camera Metric Depth Estimation](https://mlsanity.com/paper/2405.02004/) (2024)
- [Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2212.05315/) (2022)
- [Diffusion-Augmented Depth Prediction with Sparse Annotations](https://mlsanity.com/paper/2308.02283/) (2023)
- [Towards Cross-View-Consistent Self-Supervised Surround Depth Estimation](https://mlsanity.com/paper/2407.04041/) (2024)
- [Adjacent-view Transformers for Supervised Surround-view Depth Estimation](https://mlsanity.com/paper/2303.07759/) (2023)
- [DeLiRa: Self-Supervised Depth, Light, and Radiance Fields](https://mlsanity.com/paper/2304.02797/) (2023)
- [SDGE: Stereo Guided Depth Estimation for 360^circ Camera Sets](https://mlsanity.com/paper/2402.11791/) (2024)
- [Towards Scale-Aware Full Surround Monodepth with Transformers](https://mlsanity.com/paper/2407.10406/) (2024)
- [BEV-LGKD: A Unified LiDAR-Guided Knowledge Distillation Framework for BEV 3D Object Detection](https://mlsanity.com/paper/2212.00623/) (2022)
- [A Simple Framework for 3D Occupancy Estimation in Autonomous Driving](https://mlsanity.com/paper/2303.10076/) (2023)
- [Robust Self-Supervised Extrinsic Self-Calibration](https://mlsanity.com/paper/2308.02153/) (2023)
- [Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios](https://mlsanity.com/paper/2402.11826/) (2024)
- [TiGDistill-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learning Distillation](https://mlsanity.com/paper/2412.20911/) (2024)

## Abstract (arXiv)

Self-supervised monocular depth and ego-motion estimation is a promising approach to replace or supplement expensive depth sensors such as LiDAR for robotics applications like autonomous driving. However, most research in this area focuses on a single monocular camera or stereo pairs that cover only a fraction of the scene around the vehicle. In this work, we extend monocular self-supervised depth and ego-motion estimation to large-baseline multi-camera rigs. Using generalized spatio-temporal contexts, pose consistency constraints, and carefully designed photometric loss masking, we learn a single network generating dense, consistent, and scale-aware point clouds that cover the same full surround 360 degree field of view as a typical LiDAR scanner. We also propose a new scale-consistent evaluation metric more suitable to multi-camera settings. Experiments on two challenging benchmarks illustrate the benefits of our approach over strong baselines.