Enhancing Monocular Depth Estimation with Multi-Source Auxiliary Tasks
Not on the map. Why?
Abstract (arXiv)
Monocular depth estimation (MDE) is a challenging task in computer vision, often hindered by the cost and scarcity of high-quality labeled datasets. We tackle this challenge using auxiliary datasets from related vision tasks for an alternating training scheme with a shared decoder built on top of a pre-trained vision foundation model, while giving a higher weight to MDE. Through extensive experiments we demonstrate the benefits of incorporating various in-domain auxiliary datasets and tasks to improve MDE quality on average by ~11%. Our experimental analysis shows that auxiliary tasks have different impacts, confirming the importance of task selection, highlighting that quality gains are not achieved by merely adding data. Remarkably, our study reveals that using semantic segmentation datasets as Multi-Label Dense Classification (MLDC) often results in additional quality gains. Lastly, our method significantly improves the data efficiency for the considered MDE datasets, enhancing their quality while reducing their size by at least 80%. This paves the way for using auxiliary data from related tasks to improve MDE quality despite limited availability of high-quality labeled data. Code is available at https://jugit.fz-juelich.de/ias-8/mdeaux.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 50 of its 74 references that are in the map. First posted on arXiv 22 Jan 2025.
Builds on
50 of 74 references.
References · 50
- 2017Attention Is All You Needcited by 59,932
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2021Vision Transformers for Dense Predictioncited by 1,789
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
- 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
- 2017A Survey on Multi-Task Learningcited by 838
38 more
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
- 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
- 2021Carbon Emissions and Large Neural Network Trainingcited by 561
- 2018End-to-End Multi-Task Learning with Attentioncited by 533
- 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
- 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
- 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
- 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
- 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
- 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
- 2019Task2Vec: Task Embedding for Meta-Learningcited by 197
- 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
- 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 173
- 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 166
- 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
- 2022DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimationcited by 116
- 2014Deep Convolutional Neural Fields for Depth Estimation from a Single Imagecited by 106
- 2019Representation Similarity Analysis for Efficient Task taxonomy & Transfer Learningcited by 86
- 2021Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimationcited by 80
- 2018Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimationcited by 78
- 2022UViM: A Unified Modeling Approach for Vision with Learned Guiding Codescited by 77
- 2023All in Tokens: Unifying Output Space of Visual Tasks via Soft Tokencited by 52
- 2018Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotationscited by 51
- 2021Learning Multiple Dense Prediction Tasks from Partially Annotated Datacited by 42
- 2021Learning to Relate Depth and Semantics for Unsupervised Domain Adaptationcited by 32
- 2019Zero-Shot Task Transfercited by 26
- 2023Vision Transformer Adapters for Generalizable Multitask Learningcited by 14
- 2023PolyMaX: General Dense Prediction with Mask Transformercited by 14
- 2024SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Imagescited by 13
- 2022AANG: Automating Auxiliary Learningcited by 12
- 2022A Survey on RGB-D Datasetscited by 11
- 2021JUWELS Booster -- A Supercomputer for Large-Scale AI Researchcited by 10
- 2022Multi-Task Distributed Learning using Vision Transformer with Random Patch Permutationcited by 10
- 2024Efficient Multi-task Uncertainties for Joint Semantic Segmentation and Monocular Depth Estimationcited by 6
- 2022Semantics-Depth-Symbiosis: Deeply Coupled Semi-Supervised Learning of Semantics and Depthcited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.