# Manydepth2: Motion-Aware Self-Supervised Monocular Depth Estimation in Dynamic Scenes

arXiv: https://arxiv.org/abs/2312.15268 · HTML: https://mlsanity.com/paper/2312.15268/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 23 Dec 2023.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras](https://mlsanity.com/paper/1904.04998/) (2019)
- [Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction](https://mlsanity.com/paper/1803.03893/) (2018)
- [Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videos](https://mlsanity.com/paper/1811.06152/) (2018)
- [Learning Depth from Monocular Videos using Direct Methods](https://mlsanity.com/paper/1712.00175/) (2017)
- [Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance](https://mlsanity.com/paper/2007.06936/) (2020)
- [Feature-metric Loss for Self-supervised Learning of Depth and Egomotion](https://mlsanity.com/paper/2007.10603/) (2020)
- [Semantically-Guided Representation Learning for Self-Supervised Monocular Depth](https://mlsanity.com/paper/2002.12319/) (2020)
- [Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding](https://mlsanity.com/paper/1810.06125/) (2018)
- [Unsupervised Monocular Depth Learning in Dynamic Scenes](https://mlsanity.com/paper/2010.16404/) (2020)
- [Don't Forget The Past: Recurrent Depth Estimation from Monocular Video](https://mlsanity.com/paper/2001.02613/) (2020)
- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera](https://mlsanity.com/paper/2011.11814/) (2020)
- [Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency](https://mlsanity.com/paper/2102.02629/) (2021)
- [Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depth](https://mlsanity.com/paper/2203.15174/) (2022)
- [Unsupervised Adversarial Depth Estimation using Cycled Generative Networks](https://mlsanity.com/paper/1807.10915/) (2018)
- [DynPoint: Dynamic Neural Point For View Synthesis](https://mlsanity.com/paper/2310.18999/) (2023)
- [Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2110.08192/) (2021)
- [Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation](https://mlsanity.com/paper/2110.06853/) (2021)
- [Peeking Behind Objects: Layered Depth Prediction from a Single Image](https://mlsanity.com/paper/1807.08776/) (2018)
- [DevNet: Self-supervised Monocular Depth Learning via Density Volume Construction](https://mlsanity.com/paper/2209.06351/) (2022)

## Built on by (most cited first, 12 of 12)

- [AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation](https://mlsanity.com/paper/2406.11548/) (2024)
- [PAGE-4D: Disentangled pose and geometry estimation for vggt-4d perception](https://mlsanity.com/paper/2510.17568/) (2025)
- [TransDiff: Diffusion-Based Method for Manipulating Transparent Objects Using a Single RGB-D Image](https://mlsanity.com/paper/2503.12779/) (2025)
- [Stream3D: Sequential Multi-View 3D Generation via Evidential Memory](https://mlsanity.com/paper/2605.21472/) (2026)
- [RISE: Single Static Radar-based Indoor Scene Understanding](https://mlsanity.com/paper/2511.14019/) (2025)
- [CoProU-VO: Combining Projected Uncertainty for End-to-End Unsupervised Monocular Visual Odometry](https://mlsanity.com/paper/2508.00568/) (2025)
- [Discriminately Treating Motion Components Evolves Joint Depth and Ego-Motion Learning](https://mlsanity.com/paper/2511.01502/) (2025)
- [MambaLoc: Efficient Camera Localisation via State Space Model](https://mlsanity.com/paper/2408.09680/) (2024)
- [DeLightMono: Enhancing Self-Supervised Monocular Depth Estimation in Endoscopy by Decoupling Uneven Illumination](https://mlsanity.com/paper/2511.20058/) (2025)
- [U2Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation](https://mlsanity.com/paper/2604.10056/) (2026)
- [Boosting Robustness for All-Weather Self-Supervised Depth Estimation in Autonomous Driving](https://mlsanity.com/paper/2607.21526/) (2026)
- [RbFT-Net: Rectify-Before-Fuse Temporal Radar Anchors for 4D Radar-Camera Depth Completion](https://mlsanity.com/paper/2608.13102/) (2026)

## Abstract (arXiv)

Despite advancements in self-supervised monocular depth estimation, challenges persist in dynamic scenarios due to the dependence on assumptions about a static world. In this paper, we present Manydepth2, to achieve precise depth estimation for both dynamic objects and static backgrounds, all while maintaining computational efficiency. To tackle the challenges posed by dynamic content, we incorporate optical flow and coarse monocular depth to create a pseudo-static reference frame. This frame is then utilized to build a motion-aware cost volume in collaboration with the vanilla target frame. Furthermore, to improve the accuracy and robustness of the network architecture, we propose an attention-based depth network that effectively integrates information from feature maps at different resolutions by incorporating both channel and non-local attention mechanisms. Compared to methods with similar computational costs, Manydepth2 achieves a significant reduction of approximately five percent in root-mean-square error for self-supervised monocular depth estimation on the KITTI-2015 dataset. The code could be found at https://github.com/kaichen-z/Manydepth2.