# HoloFusion: Towards Photo-realistic 3D Generative Modeling

arXiv: https://arxiv.org/abs/2308.14244 · HTML: https://mlsanity.com/paper/2308.14244/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 43 of its 49 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022)
- [RealFusion: 360deg Reconstruction of Any Object from a Single Image](https://mlsanity.com/paper/2302.10663/) (2023)
- [Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2210.04628/) (2022)
- [3D Neural Field Generation using Triplane Diffusion](https://mlsanity.com/paper/2211.16677/) (2022)
- [SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstruction](https://mlsanity.com/paper/2212.00792/) (2022)
- [DiffRF: Rendering-Guided 3D Radiance Field Diffusion](https://mlsanity.com/paper/2212.01206/) (2022)

## Built on by (most cited first, 30 of 32)

- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [ReconFusion: 3D Reconstruction with Diffusion Priors](https://mlsanity.com/paper/2312.02981/) (2023)
- [RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D](https://mlsanity.com/paper/2311.16918/) (2023)
- [SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency](https://mlsanity.com/paper/2407.17470/) (2024)
- [State of the Art on Diffusion Models for Visual Computing](https://mlsanity.com/paper/2310.07204/) (2023)
- [Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models](https://mlsanity.com/paper/2312.13913/) (2023)
- [latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstruction](https://mlsanity.com/paper/2403.16292/) (2024)
- [LooseControl: Lifting ControlNet for Generalized Depth Conditioning](https://mlsanity.com/paper/2312.03079/) (2023)
- [ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models](https://mlsanity.com/paper/2403.01807/) (2024)
- [DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation](https://mlsanity.com/paper/2501.16764/) (2025)
- [SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generation](https://mlsanity.com/paper/2503.16396/) (2025)
- [PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion](https://mlsanity.com/paper/2312.09069/) (2023)
- [CAD: Photorealistic 3D Generation via Adversarial Distillation](https://mlsanity.com/paper/2312.06663/) (2023)
- [DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data](https://mlsanity.com/paper/2406.04322/) (2024)
- [4Dynamic: Text-to-4D Generation with Hybrid Priors](https://mlsanity.com/paper/2407.12684/) (2024)
- [ViewFusion: Towards Multi-View Consistency via Interpolated Denoising](https://mlsanity.com/paper/2402.18842/) (2024)
- [Lightplane: Highly-Scalable Components for Neural 3D Fields](https://mlsanity.com/paper/2404.19760/) (2024)
- [Envision3D: One Image to 3D with Anchor Views Interpolation](https://mlsanity.com/paper/2403.08902/) (2024)
- [OccFusion: Rendering Occluded Humans with Generative Diffusion Priors](https://mlsanity.com/paper/2407.00316/) (2024)
- [CharacterShot: Controllable and Consistent 4D Character Animation](https://mlsanity.com/paper/2508.07409/) (2025)
- [Wonder3D++: Cross-domain Diffusion for High-fidelity 3D Generation from a Single Image](https://mlsanity.com/paper/2511.01767/) (2025)
- [Direct and Explicit 3D Generation from a Single Image](https://mlsanity.com/paper/2411.10947/) (2024)
- [High-Quality 3D Creation from A Single Image Using Subject-Specific Knowledge Prior](https://mlsanity.com/paper/2312.11535/) (2023)
- [Part-aware Shape Generation with Latent 3D Diffusion of Neural Voxel Fields](https://mlsanity.com/paper/2405.00998/) (2024)
- [Stable Part Diffusion 4D: Multi-View RGB and Kinematic Parts Video Generation](https://mlsanity.com/paper/2509.10687/) (2025)
- [FB-4D: Spatial-Temporal Coherent Dynamic 3D Content Generation with Feature Banks](https://mlsanity.com/paper/2503.20784/) (2025)
- [GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learning](https://mlsanity.com/paper/2312.08744/) (2023)
- [GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space](https://mlsanity.com/paper/2412.16717/) (2024)
- [Pomo3D: 3D-Aware Portrait Accessorizing and More](https://mlsanity.com/paper/2409.14430/) (2024)

## Abstract (arXiv)

Diffusion-based image generators can now produce high-quality and diverse samples, but their success has yet to fully translate to 3D generation: existing diffusion methods can either generate low-resolution but 3D consistent outputs, or detailed 2D views of 3D objects but with potential structural defects and lacking view consistency or realism. We present HoloFusion, a method that combines the best of these approaches to produce high-fidelity, plausible, and diverse 3D samples while learning from a collection of multi-view 2D images only. The method first generates coarse 3D samples using a variant of the recently proposed HoloDiffusion generator. Then, it independently renders and upsamples a large number of views of the coarse 3D model, super-resolves them to add detail, and distills those into a single, high-fidelity implicit 3D representation, which also ensures view consistency of the final renders. The super-resolution network is trained as an integral part of HoloFusion, end-to-end, and the final distillation uses a new sampling scheme to capture the space of super-resolved signals. We compare our method against existing baselines, including DreamFusion, Get3D, EG3D, and HoloDiffusion, and achieve, to the best of our knowledge, the most realistic results on the challenging CO3Dv2 dataset.