# M2Depth: Self-supervised Two-Frame Multi-camera Metric Depth Estimation

arXiv: https://arxiv.org/abs/2405.02004 · HTML: https://mlsanity.com/paper/2405.02004/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 45 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://mlsanity.com/paper/2306.14289/) (2023)
- [Customized Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.13785/) (2023)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Group-wise Correlation Stereo Network](https://mlsanity.com/paper/1903.04025/) (2019)
- [Inpaint Anything: Segment Anything Meets Image Inpainting](https://mlsanity.com/paper/2304.06790/) (2023)
- [Segment and Track Anything](https://mlsanity.com/paper/2305.06558/) (2023)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation](https://mlsanity.com/paper/2204.00987/) (2022)
- [TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers](https://mlsanity.com/paper/2111.14600/) (2021)
- [Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2211.13202/) (2022)
- [Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry](https://mlsanity.com/paper/1807.02570/) (2018)
- [SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2204.03636/) (2022)
- [Bi-Directional Cascade Network for Perceptual Edge Detection](https://mlsanity.com/paper/1902.10903/) (2019)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera](https://mlsanity.com/paper/2011.11814/) (2020)
- [Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depth](https://mlsanity.com/paper/2203.15174/) (2022)
- [Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry](https://mlsanity.com/paper/2112.08177/) (2021)
- [Full Surround Monodepth from Multiple Cameras](https://mlsanity.com/paper/2104.00152/) (2021)
- [R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras](https://mlsanity.com/paper/2308.14713/) (2023)

## Built on by (most cited first, 10 of 10)

- [TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers](https://mlsanity.com/paper/2408.13770/) (2024)
- [DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generation](https://mlsanity.com/paper/2503.15208/) (2025)
- [OmniNWM: Omniscient Driving Navigation World Models](https://mlsanity.com/paper/2510.18313/) (2025)
- [LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understanding](https://mlsanity.com/paper/2412.17635/) (2024)
- [MuDG: Taming Multi-modal Diffusion with Gaussian Splatting for Urban Scene Reconstruction](https://mlsanity.com/paper/2503.10604/) (2025)
- [Semi-SMD: Semi-Supervised Metric Depth Estimation via Surrounding Cameras for Autonomous Driving](https://mlsanity.com/paper/2503.19713/) (2025)
- [SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Driving](https://mlsanity.com/paper/2606.16960/) (2026)
- [UniPR: Unified Object-level Real-to-Sim Perception and Reconstruction from a Single Stereo Pair](https://mlsanity.com/paper/2603.19616/) (2026)
- [DrivingDepth: Sparse-Prompted Pixel-wise Scale Correction for Driving Depth Estimation](https://mlsanity.com/paper/2606.31488/) (2026)
- [Boosting Robustness for All-Weather Self-Supervised Depth Estimation in Autonomous Driving](https://mlsanity.com/paper/2607.21526/) (2026)

## Abstract (arXiv)

This paper presents a novel self-supervised two-frame multi-camera metric depth estimation network, termed M2Depth, which is designed to predict reliable scale-aware surrounding depth in autonomous driving. Unlike the previous works that use multi-view images from a single time-step or multiple time-step images from a single camera, M2Depth takes temporally adjacent two-frame images from multiple cameras as inputs and produces high-quality surrounding depth. We first construct cost volumes in spatial and temporal domains individually and propose a spatial-temporal fusion module that integrates the spatial-temporal information to yield a strong volume presentation. We additionally combine the neural prior from SAM features with internal features to reduce the ambiguity between foreground and background and strengthen the depth edges. Extensive experimental results on nuScenes and DDAD benchmarks show M2Depth achieves state-of-the-art performance. More results can be found in https://heiheishuang.xyz/M2Depth .