# GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generation

arXiv: https://arxiv.org/abs/2311.17971 · HTML: https://mlsanity.com/paper/2311.17971/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 51 of its 62 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 51)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model](https://mlsanity.com/paper/2310.15110/) (2023)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [Shap-E: Generating Conditional 3D Implicit Functions](https://mlsanity.com/paper/2305.02463/) (2023)
- [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022)
- [RealFusion: 360deg Reconstruction of Any Object from a Single Image](https://mlsanity.com/paper/2302.10663/) (2023)

## Built on by (most cited first, 30 of 30)

- [You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale](https://mlsanity.com/paper/2412.06699/) (2024)
- [DiffGS: Functional Gaussian Splatting Diffusion](https://mlsanity.com/paper/2410.19657/) (2024)
- [Binocular-Guided 3D Gaussian Splatting with View Consistency for Sparse View Synthesis](https://mlsanity.com/paper/2410.18822/) (2024)
- [Zero-Shot Scene Reconstruction from Single Images with Deep Prior Assembly](https://mlsanity.com/paper/2410.15971/) (2024)
- [UDiFF: Generating Conditional Unsigned Distance Fields with Optimal Wavelet Diffusion](https://mlsanity.com/paper/2404.06851/) (2024)
- [Collaborative Control for Geometry-Conditioned PBR Image Generation](https://mlsanity.com/paper/2402.05919/) (2024)
- [Hyper-3DG: Text-to-3D Gaussian Generation via Hypergraph](https://mlsanity.com/paper/2403.09236/) (2024)
- [TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshes](https://mlsanity.com/paper/2405.20283/) (2024)
- [Fast Learning of Signed Distance Functions from Noisy Point Clouds via Noise to Noise Mapping](https://mlsanity.com/paper/2407.14225/) (2024)
- [Geometry in Style: 3D Stylization via Surface Normal Deformation](https://mlsanity.com/paper/2503.23241/) (2025)
- [Envision3D: One Image to 3D with Anchor Views Interpolation](https://mlsanity.com/paper/2403.08902/) (2024)
- [3DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Models](https://mlsanity.com/paper/2503.21745/) (2025)
- [Large Point-to-Gaussian Model for Image-to-3D Generation](https://mlsanity.com/paper/2408.10935/) (2024)
- [Dreamer XL: Towards High-Resolution Text-to-3D Generation via Trajectory Score Matching](https://mlsanity.com/paper/2405.11252/) (2024)
- [RecDreamer: Consistent Text-to-3D Generation via Uniform Score Distillation](https://mlsanity.com/paper/2502.12640/) (2025)
- [UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mapping](https://mlsanity.com/paper/2502.01846/) (2025)
- [Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data](https://mlsanity.com/paper/2503.21694/) (2025)
- [DreamBeast: Distilling 3D Fantastical Animals with Part-Aware Knowledge Transfer](https://mlsanity.com/paper/2409.08271/) (2024)
- [Flow Score Distillation for Diverse Text-to-3D Generation](https://mlsanity.com/paper/2405.10988/) (2024)
- [High-fidelity 3D Object Generation from Single Image with RGBN-Volume Gaussian Reconstruction Model](https://mlsanity.com/paper/2504.01512/) (2025)
- [Few-step Flow for 3D Generation via Marginal-Data Transport Distillation](https://mlsanity.com/paper/2509.04406/) (2025)
- [Self-Evolving 3D Scene Generation from a Single Image](https://mlsanity.com/paper/2512.08905/) (2025)
- [ModeDreamer: Mode Guiding Score Distillation for Text-to-3D Generation using Reference Image Prompts](https://mlsanity.com/paper/2411.18135/) (2024)
- [3D-WAG: Hierarchical Wavelet-Guided Autoregressive Generation for High-Fidelity 3D Shapes](https://mlsanity.com/paper/2411.19037/) (2024)
- [Text-Image Conditioned Diffusion for Consistent Text-to-3D Generation](https://mlsanity.com/paper/2312.11774/) (2023)
- [DreamPolish: Domain Score Distillation With Progressive Geometry Generation](https://mlsanity.com/paper/2411.01602/) (2024)
- [Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching](https://mlsanity.com/paper/2506.13594/) (2025)
- [Memorization in 3D Shape Generation: An Empirical Study](https://mlsanity.com/paper/2512.23628/) (2025)
- [GaussianGrow: Geometry-aware Gaussian Growing from 3D Point Clouds with Text Guidance](https://mlsanity.com/paper/2604.05721/) (2026)
- [Focus on Neighbors and Know the Whole: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creation](https://mlsanity.com/paper/2408.13149/) (2024)

## Abstract (arXiv)

Text-to-3D generation by distilling pretrained large-scale text-to-image diffusion models has shown great promise but still suffers from inconsistent 3D geometric structures (Janus problems) and severe artifacts. The aforementioned problems mainly stem from 2D diffusion models lacking 3D awareness during the lifting. In this work, we present GeoDream, a novel method that incorporates explicit generalized 3D priors with 2D diffusion priors to enhance the capability of obtaining unambiguous 3D consistent geometric structures without sacrificing diversity or fidelity. Specifically, we first utilize a multi-view diffusion model to generate posed images and then construct cost volume from the predicted image, which serves as native 3D geometric priors, ensuring spatial consistency in 3D space. Subsequently, we further propose to harness 3D geometric priors to unlock the great potential of 3D awareness in 2D diffusion priors via a disentangled design. Notably, disentangling 2D and 3D priors allows us to refine 3D geometric priors further. We justify that the refined 3D geometric priors aid in the 3D-aware capability of 2D diffusion priors, which in turn provides superior guidance for the refinement of 3D geometric priors. Our numerical and visual comparisons demonstrate that GeoDream generates more 3D consistent textured meshes with high-resolution realistic renderings (i.e., 1024 × 1024) and adheres more closely to semantic coherence.