mlsanityData to 7 Oct 2026

Monocular Depth Estimation using Diffusion Models

arXiv 2302.14816 ↗First posted 28 Feb 2023cs.CVcited by 87

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We formulate monocular depth estimation using denoising diffusion models, inspired by their recent successes in high fidelity image generation. To that end, we introduce innovations to address problems arising due to noisy, incomplete depth maps in training data, including step-unrolled denoising diffusion, an L1 loss, and depth infilling during training. To cope with the limited availability of data for supervised training, we leverage pre-training on self-supervised image-to-image translation tasks. Despite the simplicity of the approach, with a generic loss and architecture, our DepthGen model achieves SOTA performance on the indoor NYU dataset, and near SOTA results on the outdoor KITTI dataset. Further, with a multimodal posterior, DepthGen naturally represents depth ambiguity (e.g., from transparent surfaces), and its zero-shot performance combined with depth imputation, enable a simple but effective text-to-3D pipeline. Project page: https://depth-gen.github.io

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 59 of its 70 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

59 of 70 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  2. 2022Novel View Synthesis with Diffusion Models6 Oct
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 56

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
38 more
  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  3. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  4. 2022Video Diffusion Modelscited by 2,280
  5. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  6. 2021Vision Transformers for Dense Predictioncited by 1,789
  7. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  8. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  9. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  10. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  11. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  12. 2016Colorful Image Colorizationcited by 1,113
  13. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  14. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  15. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  16. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  17. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  18. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  19. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  20. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  21. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  22. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  23. 2016Learning Representations for Automatic Colorizationcited by 251
  24. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  25. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  26. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 229
  27. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  28. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 166
  29. 2022Revealing the Dark Secrets of Masked Image Modelingcited by 117
  30. 2020Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volumecited by 116
  31. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  32. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  33. 2022Attention Attention Everywhere: Monocular Depth Prediction with Skip Attentioncited by 92
  34. 2016Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networkscited by 91
  35. 2020Guiding Monocular Depth Estimation Using Depth-Attention Volumecited by 82
  36. 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
  37. 2019Does computer vision matter for action?cited by 64
  38. 2017Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagerycited by 56

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 21; 2024: 41; 2025: 21; 2026: 4.

87 papers.

Sort:

By area:

  1. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  2. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
  3. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
  4. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  5. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  6. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  7. 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Thinkcited by 82
  8. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  9. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matchingcited by 76
  10. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 55
  11. 2023LDM3D: Latent Diffusion Model for 3Dcited by 53
  12. 2023LiDAR Data Synthesis with Denoising Diffusion Probabilistic Modelscited by 53
38 more
  1. 2024Video-Guided Foley Sound Generation with Multimodal Controlscited by 49
  2. 2024Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusioncited by 47
  3. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  4. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 46
  5. 2024ECoDepth: Effective Conditioning of Diffusion Models for Monocular Depth Estimationcited by 45
  6. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 45
  7. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Modelcited by 44
  8. 2024SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentationcited by 41
  9. 2024WorDepth: Variational Language Prior for Monocular Depth Estimationcited by 34
  10. 2023Ref-Diff: Zero-shot Referring Image Segmentation with Generative Modelscited by 33
  11. 2024View Selection for 3D Captioning via Diffusion Rankingcited by 31
  12. 2023CLE Diffusion: Controllable Light Enhancement Diffusion Modelcited by 30
  13. 2023FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimatorscited by 29
  14. 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditionscited by 29
  15. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 28
  16. 2024BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular Depth Estimationcited by 28
  17. 2024Video Depth without Video Modelscited by 28
  18. 2024DepthLab: From Partial to Completecited by 27
  19. 2025Diff2Flow: Training Flow Matching Models via Diffusion Model Alignmentcited by 27
  20. 20233DiffTection: 3D Object Detection with Geometry-Aware Diffusion Featurescited by 26
  21. 2023DiffI2I: Efficient Diffusion Model for Image-to-Image Translationcited by 24
  22. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  23. 2023Diffusion Model for Dense Matchingcited by 21
  24. 2024Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketchescited by 21
  25. 2023TIAM -- A Metric for Evaluating Alignment in Text-to-Image Generationcited by 18
  26. 2024IPAdapter-Instruct: Resolving Ambiguity in Image-based Conditioning using Instruct Promptscited by 18
  27. 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimationcited by 18
  28. 2024Distillation of Diffusion Features for Semantic Correspondencecited by 17
  29. 2025Orchid: Image Latent Diffusion for Joint Appearance and Geometry Generationcited by 17
  30. 2024Scaling Properties of Diffusion Models for Perceptual Taskscited by 15
  31. 2023Harnessing Diffusion Models for Visual Perception with Meta Promptscited by 13
  32. 2024Osmosis: RGBD Diffusion Prior for Underwater Image Restorationcited by 13
  33. 2025Depth AnyEvent: A Cross-Modal Distillation Paradigm for Event-Based Monocular Depth Estimationcited by 13
  34. 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Modelcited by 12
  35. 2023DREAM: Diffusion Rectification and Estimation-Adaptive Modelscited by 12
  36. 2025DreamCube: 3D Panorama Generation via Multi-plane Synchronizationcited by 12
  37. 2023Diffusion for Natural Image Mattingcited by 11
  38. 2024IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generationcited by 11

50 most cited of 87 All 87 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation
  2. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation
  3. 2026Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
  4. 2023Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model
  5. 2024Iris: Integrating Language into Diffusion-based Monocular Depth Estimation
  6. 2022DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models
  7. 2026Iris: Bringing Real-World Priors into Diffusion Model for Monocular Depth Estimation
  8. 2024DepthART: Monocular Depth Estimation as Autoregressive Refinement Task