mlsanityData to 7 Oct 2026

Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions

arXiv 2407.16698 ↗First posted 23 Jul 2024cs.CVcited by 29

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present a novel approach designed to address the complexities posed by challenging, out-of-distribution data in the single-image depth estimation task. Starting with images that facilitate depth prediction due to the absence of unfavorable factors, we systematically generate new, user-defined scenes with a comprehensive set of challenges and associated depth information. This is achieved by leveraging cutting-edge text-to-image diffusion models with depth-aware control, known for synthesizing high-quality image content from textual prompts while preserving the coherence of 3D structure between generated and source imagery. Subsequent fine-tuning of any monocular depth network is carried out through a self-distillation protocol that takes into account images generated using our strategy and its own depth predictions on simple, unchallenging scenes. Experiments on benchmarks tailored for our purposes demonstrate the effectiveness and versatility of our proposal.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 88 of its 107 references that are in the map. First posted on arXiv 23 Jul 2024.

Builds on

88 of 107 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation2 Jun
  2. 2023Monocular Depth Estimation using Diffusion Models28 Feb

Other references · 86

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  10. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  12. 2021Vision Transformers for Dense Predictioncited by 1,789
38 more
  1. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  2. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  4. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  5. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  6. 2021Variational Diffusion Modelscited by 1,145
  7. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  8. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  9. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  10. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  11. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  12. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  13. 2017Generative Adversarial Networks: An Overviewcited by 588
  14. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  15. 2023Zero-shot Image-to-Image Translationcited by 503
  16. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  17. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  18. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  19. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  20. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  21. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 358
  22. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  23. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  24. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  25. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  26. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  27. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  28. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  29. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  30. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  31. 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
  32. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  33. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  34. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  35. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  36. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  37. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  38. 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 151

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 16; 2026: 10.

29 papers.

Sort:

By area:

  1. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  2. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  3. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 45
  4. 2025Synthetic-to-Real Self-supervised Robust Depth Estimation via Learning with Motion and Structure Priorscited by 20
  5. 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimationcited by 18
  6. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  7. 2025Depth Anything at Any Conditioncited by 11
  8. 2025Seeing and Seeing Through the Glass: Real and Synthetic Data for Multi-Layer Depth Estimationcited by 8
  9. 2025The Fourth Monocular Depth Estimation Challengecited by 8
  10. 2025Always Clear Depth: Robust Monocular Depth Estimation under Adverse Weathercited by 4
  11. 2026Diffusion Model as a Generalist Segmentation Learnercited by 4
  12. 2025NTIRE 2025 Challenge on HR Depth from Images of Specular and Transparent Surfacescited by 3
17 more
  1. 2025Control and Realism: Best of Both Worlds in Layout-to-Image without Trainingcited by 3
  2. 2025Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusioncited by 2
  3. 2025ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimationcited by 2
  4. 2025Learning Depth from Past Selves: Self-Evolution Contrast for Robust Depth Estimationcited by 2
  5. 2026Symmetry Informative and Agnostic Feature Disentanglement for 3D Shapescited by 2
  6. 2026SeeClear: Reliable Transparent Object Depth Estimation via Generative Opacificationcited by 2
  7. 2025Perfecting Depth: Uncertainty-Aware Enhancement of Metric Depthcited by 1
  8. 2025EGSA-PT:Edge-Guided Spatial Attention with Progressive Training for Monocular Depth Estimation and Segmentation of Transparent Objectscited by 1
  9. 2026ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Devicecited by 1
  10. 2024Multi-Modality Driven LoRA for Adverse Condition Depth Estimationcited by 0
  11. 2025Survey on Monocular Metric Depth Estimationcited by 0
  12. 2026SeeGroup: Multi-Layer Depth Estimation of Transparent Surfaces via Self-Determined Groupingcited by 0
  13. 2026Boosting Robustness for All-Weather Self-Supervised Depth Estimation in Autonomous Drivingcited by 0
  14. 2026GIFT: Geometry-Invariant Fine-Tuning for Non-Lambertian Monocular Depth Estimationcited by 0
  15. 2026Monocular Depth Estimation from a Single Image: Progress and Opportunitiescited by 0
  16. 2026Weather-Conditioned Depth Anythingcited by 0
  17. 2026Depth-to-Image Synthesis-Driven Generative Unguided Depth Completioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Iris: Integrating Language into Diffusion-based Monocular Depth Estimation
  2. 2026Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
  3. 2025Leveraging Stable Diffusion for Monocular Depth Estimation via Image Semantic Encoding
  4. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation
  5. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matching
  6. 2024Unsupervised Monocular Depth Estimation Based on Hierarchical Feature-Guided Diffusion
  7. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers
  8. 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Model