# MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera

arXiv: https://arxiv.org/abs/2011.11814 · HTML: https://mlsanity.com/paper/2011.11814/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2020.

## Built on by (most cited first, 30 of 59)

- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [Behind the Scenes: Density Fields for Single View Reconstruction](https://mlsanity.com/paper/2301.07668/) (2023)
- [Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches](https://mlsanity.com/paper/2207.04718/) (2022)
- [STS: Surround-view Temporal Stereo for Multi-view 3D Detection](https://mlsanity.com/paper/2208.10145/) (2022)
- [Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depth](https://mlsanity.com/paper/2203.15174/) (2022)
- [TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo](https://mlsanity.com/paper/2111.07418/) (2021)
- [R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras](https://mlsanity.com/paper/2308.14713/) (2023)
- [Self-supervised Monocular Depth Estimation: Let's Talk About The Weather](https://mlsanity.com/paper/2307.08357/) (2023)
- [Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving](https://mlsanity.com/paper/2403.07535/) (2024)
- [S4C: Self-Supervised Semantic Scene Completion with Neural Fields](https://mlsanity.com/paper/2310.07522/) (2023)
- [Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2110.08192/) (2021)
- [Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmark](https://mlsanity.com/paper/2212.08914/) (2022)
- [MVSAnywhere: Zero-Shot Multi-View Stereo](https://mlsanity.com/paper/2503.22430/) (2025)
- [Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes](https://mlsanity.com/paper/2304.08993/) (2023)
- [MACARONS: Mapping And Coverage Anticipation with RGB Online Self-Supervision](https://mlsanity.com/paper/2303.03315/) (2023)
- [Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks](https://mlsanity.com/paper/2301.13487/) (2023)
- [CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstruction](https://mlsanity.com/paper/2304.14633/) (2023)
- [Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning](https://mlsanity.com/paper/2208.09170/) (2022)
- [SAAM: Stealthy Adversarial Attack on Monocular Depth Estimation](https://mlsanity.com/paper/2308.03108/) (2023)
- [Manydepth2: Motion-Aware Self-Supervised Monocular Depth Estimation in Dynamic Scenes](https://mlsanity.com/paper/2312.15268/) (2023)
- [DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstruction](https://mlsanity.com/paper/2412.04464/) (2024)
- [APARATE: Adaptive Adversarial Patch for CNN-based Monocular Depth Estimation for Autonomous Navigation](https://mlsanity.com/paper/2303.01351/) (2023)
- [M2Depth: Self-supervised Two-Frame Multi-camera Metric Depth Estimation](https://mlsanity.com/paper/2405.02004/) (2024)
- [SPEAL: Skeletal Prior Embedded Attention Learning for Cross-Source Point Cloud Registration](https://mlsanity.com/paper/2312.08664/) (2023)
- [Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2212.05315/) (2022)
- [DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volume](https://mlsanity.com/paper/2308.07225/) (2023)
- [DurLAR: A High-fidelity 128-channel LiDAR Dataset with Panoramic Ambient and Reflectivity Imagery for Multi-modal Autonomous Driving Applications](https://mlsanity.com/paper/2406.10068/) (2024)
- [A Low Memory Footprint Quantized Neural Network for Depth Completion of Very Sparse Time-of-Flight Depth Maps](https://mlsanity.com/paper/2205.12918/) (2022)

## Abstract (arXiv)

In this paper, we propose MonoRec, a semi-supervised monocular dense reconstruction architecture that predicts depth maps from a single moving camera in dynamic environments. MonoRec is based on a multi-view stereo setting which encodes the information of multiple consecutive images in a cost volume. To deal with dynamic objects in the scene, we introduce a MaskModule that predicts moving object masks by leveraging the photometric inconsistencies encoded in the cost volumes. Unlike other multi-view stereo methods, MonoRec is able to reconstruct both static and moving objects by leveraging the predicted masks. Furthermore, we present a novel multi-stage training scheme with a semi-supervised loss formulation that does not require LiDAR depth values. We carefully evaluate MonoRec on the KITTI dataset and show that it achieves state-of-the-art performance compared to both multi-view and single-view methods. With the model trained on KITTI, we further demonstrate that MonoRec is able to generalize well to both the Oxford RobotCar dataset and the more challenging TUM-Mono dataset recorded by a handheld camera. Code and related materials will be available at https://vision.in.tum.de/research/monorec.