mlsanityData to 14 Oct 2026

DDP: Diffusion Model for Dense Visual Prediction

arXiv 2303.17559 ↗First posted 30 Mar 2023cs.CVcited by 148

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We propose a simple, efficient, yet powerful framework for dense visual predictions based on the conditional diffusion pipeline. Our approach follows a "noise-to-map" generative paradigm for prediction by progressively removing noise from a random Gaussian distribution, guided by the image. The method, called DDP, efficiently extends the denoising diffusion process into the modern perception pipeline. Without task-specific design and architecture customization, DDP is easy to generalize to most dense prediction tasks, e.g., semantic segmentation and depth estimation. In addition, DDP shows attractive properties such as dynamic inference and uncertainty awareness, in contrast to previous single-step discriminative methods. We show top results on three representative tasks with six diverse benchmarks, without tricks, DDP achieves state-of-the-art or competitive performance on each task compared to the specialist counterparts. For example, semantic segmentation (83.9 mIoU on Cityscapes), BEV map segmentation (70.6 mIoU on nuScenes), and depth estimation (0.05 REL on KITTI). We hope that our approach will serve as a solid baseline and facilitate future research

Key facts. Cited by 148 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 89 of its 98 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

89 of 98 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  2. 2022DiffusionDet: Diffusion Model for Object Detection17 Nov
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 84

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  10. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  11. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
38 more
  1. 2022A ConvNet for the 2020scited by 3,579
  2. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  3. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  4. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  6. 2016Improved Techniques for Training GANscited by 2,970
  7. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  9. 2016Pyramid Scene Parsing Networkcited by 2,479
  10. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  11. 2022Video Diffusion Modelscited by 2,305
  12. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  13. 2021Vision Transformers for Dense Predictioncited by 1,803
  14. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  15. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  16. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  17. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  18. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  19. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  20. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  21. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  22. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  23. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  24. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  25. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  26. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  27. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  28. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  29. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  30. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 576
  31. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  32. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  33. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  34. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 445
  35. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
  36. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 373
  37. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  38. 2022DiffDock: Diffusion Steps, Twists, and Turns for Molecular Dockingcited by 313

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 28; 2024: 58; 2025: 46; 2026: 16.

148 papers.

Sort:

By area:

  1. 2024Depth Anything V2cited by 1,422
  2. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  3. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 327
  4. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 286
  5. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
  6. 2023A Survey on Video Diffusion Modelscited by 206
  7. 2024StableNormal: Reducing Diffusion Variance for Stable and Sharp Normalcited by 154
  8. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  9. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  10. 2023InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Languagecited by 96
  11. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matchingcited by 76
  12. 2023GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generationcited by 56
38 more
  1. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 56
  2. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 51
  3. 2024Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusioncited by 50
  4. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 47
  5. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Modelcited by 45
  6. 2024ECoDepth: Effective Conditioning of Diffusion Models for Monocular Depth Estimationcited by 45
  7. 2023Interpretable Diffusion via Information Decompositioncited by 37
  8. 2023Towards Generic Semi-Supervised Framework for Volumetric Medical Image Segmentationcited by 35
  9. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  10. 2023Ref-Diff: Zero-shot Referring Image Segmentation with Generative Modelscited by 33
  11. 2023Contrastive Conditional Latent Diffusion for Audio-visual Segmentationcited by 31
  12. 2023SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Processcited by 29
  13. 2023Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Modelcited by 29
  14. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 29
  15. 2024BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular Depth Estimationcited by 28
  16. 2023Diffusion Model for Camouflaged Object Detectioncited by 27
  17. 2024DepthLab: From Partial to Completecited by 27
  18. 2024Not All Diffusion Model Activations Have Been Evaluated as Discriminative Featurescited by 25
  19. 2023DiffI2I: Efficient Diffusion Model for Image-to-Image Translationcited by 24
  20. 2023Diffusion Model for Dense Matchingcited by 21
  21. 2024Diff-IP2D: Diffusion-Based Hand-Object Interaction Prediction on Egocentric Videoscited by 21
  22. 2025Jasmine: Harnessing Diffusion Prior for Self-supervised Depth Estimationcited by 21
  23. 2023DFormer: Diffusion-guided Transformer for Universal Image Segmentationcited by 19
  24. 2024Digging into contrastive learning for robust depth estimation with diffusion modelscited by 19
  25. 2025DepthMaster: Taming Diffusion Models for Monocular Depth Estimationcited by 19
  26. 2023SegGen: Supercharging Segmentation Models with Text2Mask and Mask2Img Synthesiscited by 18
  27. 2024A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpaintingcited by 18
  28. 2025Orchid: Image Latent Diffusion for Joint Appearance and Geometry Generationcited by 17
  29. 2023DiffusionTalker: Personalization and Acceleration for Speech-Driven 3D Face Diffusercited by 16
  30. 2024ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimationcited by 15
  31. 2024Scaling Properties of Diffusion Models for Perceptual Taskscited by 15
  32. 2025Depth AnyEvent: A Cross-Modal Distillation Paradigm for Event-Based Monocular Depth Estimationcited by 15
  33. 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Modelcited by 13
  34. 2023Harnessing Diffusion Models for Visual Perception with Meta Promptscited by 13
  35. 2024Improving Bird's Eye View Semantic Segmentation by Task Decompositioncited by 13
  36. 2023DREAM: Diffusion Rectification and Estimation-Adaptive Modelscited by 12
  37. 2024BLOS-BEV: Navigation Map Enhanced Lane Segmentation Network, Beyond Line of Sightcited by 12
  38. 2024Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Modelscited by 12

50 most cited of 148 All 148 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Models in Low-Level Vision: A Survey
  2. 2024DPBridge: Latent Diffusion Bridge for Dense Prediction
  3. 2025D3-Predictor: Noise-Free Deterministic Diffusion for Dense Prediction
  4. 2023Exploiting Diffusion Prior for Generalizable Dense Prediction
  5. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Prediction
  6. 2024Toward a Diffusion-Based Generalist for Dense Vision Tasks
  7. 2024What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?
  8. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation