# Multi-Camera Collaborative Depth Prediction via Consistent Structure Estimation

arXiv: https://arxiv.org/abs/2210.02009 · HTML: https://mlsanity.com/paper/2210.02009/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 42 references that are in the map.
- First posted on arXiv 5 Oct 2022.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [Enforcing geometric constraints of virtual normal for depth prediction](https://mlsanity.com/paper/1907.12209/) (2019)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields](https://mlsanity.com/paper/1502.07411/) (2015)
- [A Survey of Structure from Motion](https://mlsanity.com/paper/1701.08493/) (2017)
- [Learning Depth from Monocular Videos using Direct Methods](https://mlsanity.com/paper/1712.00175/) (2017)
- [Cost Volume Pyramid Based Depth Inference for Multi-View Stereo](https://mlsanity.com/paper/1912.08329/) (2019)
- [CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://mlsanity.com/paper/1804.00874/) (2018)
- [HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching](https://mlsanity.com/paper/2007.12140/) (2020)
- [SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2204.03636/) (2022)
- [Normal Assisted Stereo Depth Estimation](https://mlsanity.com/paper/1911.10444/) (2019)
- [Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction](https://mlsanity.com/paper/2103.04216/) (2021)
- [Full Surround Monodepth from Multiple Cameras](https://mlsanity.com/paper/2104.00152/) (2021)
- [Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency](https://mlsanity.com/paper/1905.02706/) (2019)
- [Bi3D: Stereo Depth Estimation via Binary Classifications](https://mlsanity.com/paper/2005.07274/) (2020)
- [Learning Depth from Single Images with Deep Neural Network Embedding Focal Length](https://mlsanity.com/paper/1803.10039/) (2018)
- [Weakly-Supervised Monocular Depth Estimationwith Resolution-Mismatched Data](https://mlsanity.com/paper/2109.11573/) (2021)

## Built on by (most cited first, 16 of 16)

- [OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments](https://mlsanity.com/paper/2312.09243/) (2023)
- [R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras](https://mlsanity.com/paper/2308.14713/) (2023)
- [MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models](https://mlsanity.com/paper/2503.08007/) (2025)
- [Sparse2DGS: Geometry-Prioritized Gaussian Splatting for Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2504.20378/) (2025)
- [Diffusion-Augmented Depth Prediction with Sparse Annotations](https://mlsanity.com/paper/2308.02283/) (2023)
- [RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion](https://mlsanity.com/paper/2306.03584/) (2023)
- [Towards Cross-View-Consistent Self-Supervised Surround Depth Estimation](https://mlsanity.com/paper/2407.04041/) (2024)
- [Adjacent-view Transformers for Supervised Surround-view Depth Estimation](https://mlsanity.com/paper/2303.07759/) (2023)
- [SDGE: Stereo Guided Depth Estimation for 360^circ Camera Sets](https://mlsanity.com/paper/2402.11791/) (2024)
- [Towards Scale-Aware Full Surround Monodepth with Transformers](https://mlsanity.com/paper/2407.10406/) (2024)
- [Robust Self-Supervised Extrinsic Self-Calibration](https://mlsanity.com/paper/2308.02153/) (2023)
- [Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios](https://mlsanity.com/paper/2402.11826/) (2024)
- [BEVScope: Enhancing Self-Supervised Depth Estimation Leveraging Bird's-Eye-View in Dynamic Scenarios](https://mlsanity.com/paper/2306.11598/) (2023)
- [DVLO4D: Deep Visual-Lidar Odometry with Sparse Spatial-temporal Fusion](https://mlsanity.com/paper/2509.06023/) (2025)
- [Multi-Frame Self-Supervised Depth Estimation with Multi-Scale Feature Fusion in Dynamic Scenes](https://mlsanity.com/paper/2303.14628/) (2023)
- [Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions](https://mlsanity.com/paper/2411.03638/) (2024)

## Abstract (arXiv)

Depth map estimation from images is an important task in robotic systems. Existing methods can be categorized into two groups including multi-view stereo and monocular depth estimation. The former requires cameras to have large overlapping areas and sufficient baseline between cameras, while the latter that processes each image independently can hardly guarantee the structure consistency between cameras. In this paper, we propose a novel multi-camera collaborative depth prediction method that does not require large overlapping areas while maintaining structure consistency between cameras. Specifically, we formulate the depth estimation as a weighted combination of depth basis, in which the weights are updated iteratively by a refinement network driven by the proposed consistency loss. During the iterative update, the results of depth estimation are compared across cameras and the information of overlapping areas is propagated to the whole depth maps with the help of basis formulation. Experimental results on DDAD and NuScenes datasets demonstrate the superior performance of our method.