mlsanityData to 7 Oct 2026

RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjects

arXiv 2307.15988 ↗First posted 29 Jul 2023cs.CV cs.LGcited by 1

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present RGB-D-Fusion, a multi-modal conditional denoising diffusion probabilistic model to generate high resolution depth maps from low-resolution monocular RGB images of humanoid subjects. RGB-D-Fusion first generates a low-resolution depth map using an image conditioned denoising diffusion probabilistic model and then upsamples the depth map using a second denoising diffusion probabilistic model conditioned on a low-resolution RGB-D image. We further introduce a novel augmentation technique, depth noise augmentation, to increase the robustness of our super-resolution model.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 83 of its 102 references that are in the map. First posted on arXiv 29 Jul 2023.

Builds on

83 of 102 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Monocular Depth Estimation using Diffusion Models28 Feb
  2. 2022DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models17 Dec
  3. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  4. 2022Novel View Synthesis with Diffusion Models6 Oct
  5. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  6. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 77

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
38 more
  1. 2022Classifier-Free Diffusion Guidancecited by 5,717
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2016Layer Normalizationcited by 5,579
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2016Density estimation using Real NVPcited by 2,029
  13. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  14. 2018Group Normalizationcited by 1,514
  15. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  16. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  17. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  18. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  19. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  20. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  21. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  22. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  23. 2021Variational Diffusion Modelscited by 1,145
  24. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  25. 2016Deep Networks with Stochastic Depthcited by 973
  26. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  27. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  28. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  29. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  30. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  31. 2019Neural Spline Flowscited by 590
  32. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  33. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  34. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  35. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  36. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  37. 2020UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentationcited by 331
  38. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 306

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024A Comprehensive Survey on Diffusion Models and Their Applicationscited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformers
  2. 2024D4D: An RGBD diffusion model to boost monocular depth estimation
  3. 2025TransDiff: Diffusion-Based Method for Manipulating Transparent Objects Using a Single RGB-D Image
  4. 2025DidSee: Diffusion-Based Depth Completion for Material-Agnostic Robotic Perception and Manipulation
  5. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation
  6. 2024DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans
  7. 2022Bi-Noising Diffusion: Towards Conditional Diffusion Models with Generative Restoration Priors
  8. 2024Coherent 3D Scene Diffusion From a Single RGB Image