mlsanityData to 7 Oct 2026

GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image

arXiv 2403.12013 ↗First posted 18 Mar 2024cs.CVcited by 283

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes, e.g., depth and normals, from single images. While significant research has already been conducted in this area, the progress has been substantially limited by the low diversity and poor quality of publicly available datasets. As a result, the prior works either are constrained to limited scenarios or suffer from the inability to capture geometric details. In this paper, we demonstrate that generative models, as opposed to traditional discriminative models (e.g., CNNs and Transformers), can effectively address the inherently ill-posed problem. We further show that leveraging diffusion priors can markedly improve generalization, detail preservation, and efficiency in resource usage. Specifically, we extend the original stable diffusion model to jointly predict depth and normal, allowing mutual information exchange and high consistency between the two representations. More importantly, we propose a simple yet effective strategy to segregate the complex data distribution of various scenes into distinct sub-distributions. This strategy enables our model to recognize different scene layouts, capturing 3D geometry with remarkable fidelity. GeoWizard sets new benchmarks for zero-shot depth and normal prediction, significantly enhancing many downstream applications such as 3D reconstruction, 2D content creation, and novel viewpoint synthesis.

Key facts. Cited by 283 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 69 of its 80 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

69 of 80 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D28 Nov
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  3. 2023HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion12 Oct
  4. 2023JointNet: Extending Text-to-Image Diffusion for Dense Distribution Modeling10 Oct
  5. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  6. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation2 Jun
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar

Other references · 62

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2023Segment Anythingcited by 8,158
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
38 more
  1. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2021Vision Transformers for Dense Predictioncited by 1,789
  5. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  7. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  8. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  9. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  10. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  11. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  12. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  13. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  14. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  15. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  16. 2020Virtual KITTI 2cited by 345
  17. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  18. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  19. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  20. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  21. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
  22. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  23. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  24. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 214
  25. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  26. 2021Neural Rays for Occlusion-aware Image-based Renderingcited by 192
  27. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  28. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 144
  29. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 141
  30. 2018Evaluation of CNN-based Single-Image Depth Estimation Methodscited by 137
  31. 20193D Ken Burns Effect from a Single Imagecited by 128
  32. 2022NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priorscited by 125
  33. 2022Neural 3D Scene Reconstruction with the Manhattan-world Assumptioncited by 123
  34. 2024Rethinking Inductive Biases for Surface Normal Estimationcited by 122
  35. 20223D Common Corruptions and Data Augmentationcited by 120
  36. 2021Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimationcited by 116
  37. 2019Neural Inverse Rendering of an Indoor Scene from a Single Imagecited by 101
  38. 2018InverseRenderNet: Learning single image inverse renderingcited by 100

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 57; 2025: 139; 2026: 87.

283 papers.

Sort:

By area:

  1. 2024Depth Anything V2cited by 1,385
  2. 2025Qwen-Image Technical Reportcited by 973
  3. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  4. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  5. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
  6. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  7. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  8. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  9. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 163
  10. 2024StableNormal: Reducing Diffusion Variance for Stable and Sharp Normalcited by 153
  11. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  12. 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridgingcited by 95
38 more
  1. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  2. 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Thinkcited by 82
  3. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  4. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  5. 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 73
  6. 20243DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generationcited by 71
  7. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  8. 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 60
  9. 2024Depth Any Video with Scalable Synthetic Datacited by 57
  10. 2024MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Modelcited by 55
  11. 2025DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Modelscited by 55
  12. 2024MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priorscited by 51
  13. 2024VCR-GauS: View Consistent Depth-Normal Regularizer for Gaussian Surface Reconstructioncited by 50
  14. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generationcited by 49
  15. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  16. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 45
  17. 2025Dens3R: A Foundation Model for 3D Geometry Predictioncited by 44
  18. 2024Instant3dit: Multiview Inpainting for Fast Editing of 3D Objectscited by 43
  19. 2025GENMO: A GENeralist Model for Human MOtioncited by 43
  20. 2025DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generationcited by 39
  21. 2024VistaDream: Sampling multiview consistent images for single-view scene reconstructioncited by 38
  22. 2025UniRelight: Learning Joint Decomposition and Synthesis for Video Relightingcited by 38
  23. 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
  24. 2024StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videoscited by 36
  25. 2025LBM: Latent Bridge Matching for Fast Image-to-Image Translationcited by 36
  26. 2025GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priorscited by 36
  27. 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worldscited by 34
  28. 2025Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Controlcited by 33
  29. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 33
  30. 2024BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular Depth Estimationcited by 28
  31. 2024Video Depth without Video Modelscited by 28
  32. 2024OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integrationcited by 28
  33. 2024DepthLab: From Partial to Completecited by 27
  34. 2025UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interfacecited by 27
  35. 2025Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Videocited by 27
  36. 2025Diff2Flow: Training Flow Matching Models via Diffusion Model Alignmentcited by 27
  37. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 26
  38. 2025UAVScenes: A Multi-Modal Dataset for UAVscited by 26

50 most cited of 283 All 283 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generation
  2. 2025G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior
  3. 2025GeoComplete: Geometry-Aware Diffusion for Reference-Driven Image Completion
  4. 2026GeoStereo: A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal
  5. 2026GeoQuery: Geometry-Query Diffusion for Sparse-View Reconstruction
  6. 2026GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis
  7. 2025Geometry-Aware Diffusion Models for Multiview Scene Inpainting
  8. 2026Repurposing Geometric Foundation Models for Multi-view Diffusion