mlsanityData to 14 Oct 2026

HiFi-123: Towards High-fidelity One Image to 3D Content Generation

arXiv 2310.06744 ↗First posted 10 Oct 2023cs.CVcited by 34

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Recent advances in diffusion models have enabled 3D generation from a single image. However, current methods often produce suboptimal results for novel views, with blurred textures and deviations from the reference image, limiting their practical applications. In this paper, we introduce HiFi-123, a method designed for high-fidelity and multi-view consistent 3D generation. Our contributions are twofold: First, we propose a Reference-Guided Novel View Enhancement (RGNV) technique that significantly improves the fidelity of diffusion-based zero-shot novel view synthesis methods. Second, capitalizing on the RGNV, we present a novel Reference-Guided State Distillation (RGSD) loss. When incorporated into the optimization-based image-to-3D pipeline, our method significantly improves 3D generation quality, achieving state-of-the-art performance. Comprehensive evaluations demonstrate the effectiveness of our approach over existing methods, both qualitatively and quantitatively. Video results are available on the project page.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 49 of its 60 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

49 of 60 references.

Closest ancestors · 16

Same direction, newest first

  1. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  2. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  3. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis22 Aug
  4. 2023HD-Fusion: Detailed Text-to-3D Generation Leveraging Multiple Noise Estimation30 Jul
  5. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  6. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  8. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  9. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  10. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  11. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors6 Dec
  12. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
4 more
  1. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  2. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  3. 2022Novel View Synthesis with Diffusion Models6 Oct
  4. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,288
21 more
  1. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  5. 2021Vision Transformers for Dense Predictioncited by 1,803
  6. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  7. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  8. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  9. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  10. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  11. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  12. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  13. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  14. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  15. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  16. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  17. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
  18. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 327
  19. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  20. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 212
  21. 2018The Contextual Loss for Image Transformation with Non-Aligned Datacited by 135

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 23; 2025: 4; 2026: 3.

34 papers.

Sort:

By area:

  1. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  2. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  3. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 117
  4. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  5. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
  6. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  7. 2024Advances in 3D Generation: A Surveycited by 63
  8. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 62
  9. 2024EvaGaussians: Event Stream Assisted Gaussian Splatting from Blurry Imagescited by 56
  10. 2024MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Modelcited by 55
  11. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  12. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
22 more
  1. 2023DreamControl: Control-Based Text-to-3D Generation with 3D Self-Priorcited by 39
  2. 2023VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encodercited by 39
  3. 2024Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cyclecited by 36
  4. 2023Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repaintingcited by 25
  5. 2024HoloDreamer: Holistic 3D Panoramic World Generation from Text Descriptionscited by 24
  6. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusioncited by 22
  7. 2024Mani-GS: Gaussian Splatting Manipulation with Triangular Meshcited by 22
  8. 2025AE-NeRF: Augmenting Event-Based Neural Radiance Fields for Non-ideal Conditions and Larger Scenecited by 19
  9. 2024DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car Reconstructioncited by 16
  10. 20244Dynamic: Text-to-4D Generation with Hybrid Priorscited by 15
  11. 2024Envision3D: One Image to 3D with Anchor Views Interpolationcited by 12
  12. 2024Edify 3D: Scalable High-Quality 3D Asset Generationcited by 10
  13. 2024RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturingcited by 10
  14. 20243DGCQA: A Quality Assessment Database for 3D AI-Generated Contentscited by 6
  15. 2024DissolveStereo: Coarse Depth Injection for Zero-Shot Stereo Video Generationcited by 6
  16. 2025GS2E: Gaussian Splatting is an Effective Data Generator for Event Stream Generationcited by 6
  17. 2024Meta 3D Gencited by 3
  18. 20254DVD: Cascaded Dense-view Video Diffusion Model for High-quality 4D Content Generationcited by 3
  19. 2026UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Modelscited by 3
  20. 2026SpatialCrafter: Single Image World Modeling with Generative 3D Proxiescited by 3
  21. 2024Challenges and Opportunities in 3D Content Generationcited by 1
  22. 2026VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priorscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Models
  2. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion
  3. 2025VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video Diffusion Models
  4. 2025High-Fidelity Novel View Synthesis via Splatting-Guided Diffusion
  5. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model
  6. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
  7. 2026RefAny3D: 3D Asset-Referenced Diffusion Models for Image Generation
  8. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models