# Semi-SMD: Semi-Supervised Metric Depth Estimation via Surrounding Cameras for Autonomous Driving

arXiv: https://arxiv.org/abs/2503.19713 · HTML: https://mlsanity.com/paper/2503.19713/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 32 of its 36 references that are in the map.
- First posted on arXiv 25 Mar 2025.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://mlsanity.com/paper/2306.14289/) (2023)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [High Quality Monocular Depth Estimation via Transfer Learning](https://mlsanity.com/paper/1812.11941/) (2018)
- [Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives](https://mlsanity.com/paper/2303.09824/) (2023)
- [Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving](https://mlsanity.com/paper/2403.04133/) (2024)
- [Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry](https://mlsanity.com/paper/1807.02570/) (2018)
- [SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2204.03636/) (2022)
- [MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera](https://mlsanity.com/paper/2011.11814/) (2020)
- [SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Driving](https://mlsanity.com/paper/2404.06892/) (2024)
- [MobileSAMv2: Faster Segment Anything to Everything](https://mlsanity.com/paper/2312.09579/) (2023)
- [Full Surround Monodepth from Multiple Cameras](https://mlsanity.com/paper/2104.00152/) (2021)
- [R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras](https://mlsanity.com/paper/2308.14713/) (2023)
- [Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes](https://mlsanity.com/paper/2304.08993/) (2023)
- [EGA-Depth: Efficient Guided Attention for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2304.03369/) (2023)
- [Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning](https://mlsanity.com/paper/2208.09170/) (2022)

## Built on by (most cited first, 1 of 1)

- [SurroundNEXO: Ego-Centric Metric Bridging for Spatially Consistent Geometry in Autonomous Driving](https://mlsanity.com/paper/2606.16960/) (2026)

## Abstract (arXiv)

In this paper, we introduce Semi-SMD, a novel metric depth estimation framework tailored for surrounding cameras equipment in autonomous driving. In this work, the input data consists of adjacent surrounding frames and camera parameters. We propose a unified spatial-temporal-semantic fusion module to construct the visual fused features. Cross-attention components for surrounding cameras and adjacent frames are utilized to focus on metric scale information refinement and temporal feature matching. Building on this, we propose a pose estimation framework using surrounding cameras, their corresponding estimated depths, and extrinsic parameters, which effectively address the scale ambiguity in multi-camera setups. Moreover, semantic world model and monocular depth estimation world model are integrated to supervised the depth estimation, which improve the quality of depth estimation. We evaluate our algorithm on DDAD and nuScenes datasets, and the results demonstrate that our method achieves state-of-the-art performance in terms of surrounding camera based depth estimation quality. The source code will be available on https://github.com/xieyuser/Semi-SMD.