mlsanityData to 14 Oct 2026

HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D

arXiv 2312.15980 ↗First posted 26 Dec 2023cs.CV cs.AIcited by 28

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Recent progress in single-image 3D generation highlights the importance of multi-view coherency, leveraging 3D priors from large-scale diffusion models pretrained on Internet-scale images. However, the aspect of novel-view diversity remains underexplored within the research landscape due to the ambiguity in converting a 2D image into 3D content, where numerous potential shapes can emerge. Here, we aim to address this research gap by simultaneously addressing both consistency and diversity. Yet, striking a balance between these two aspects poses a considerable challenge due to their inherent trade-offs. This work introduces HarmonyView, a simple yet effective diffusion sampling technique adept at decomposing two intricate aspects in single-image 3D generation: consistency and diversity. This approach paves the way for a more nuanced exploration of the two critical dimensions within the sampling process. Moreover, we propose a new evaluation metric based on CLIP image and text encoders to comprehensively assess the diversity of the generated views, which closely aligns with human evaluators' judgments. In experiments, HarmonyView achieves a harmonious balance, demonstrating a win-win scenario in both consistency and diversity.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 79 of its 91 references that are in the map. First posted on arXiv 26 Dec 2023.

Builds on

79 of 91 references.

Closest ancestors · 26

Same direction, newest first

  1. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  3. 2023Consistent123: Improve Consistency for One Image to 3D Object Synthesis12 Oct
  4. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models4 Oct
  5. 2023Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day4 Oct
  6. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priors29 Sep
  7. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  8. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  9. 2023Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views27 Aug
  10. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  11. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  12. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data13 Jun
14 more
  1. 2023ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image25 May
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  3. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  4. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  5. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  7. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation9 Mar
  8. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  9. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  10. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov
  11. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  12. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  13. 2022Novel View Synthesis with Diffusion Models6 Oct
  14. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
38 more
  1. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  3. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  5. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  6. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  7. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  8. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  9. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  10. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  11. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  12. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  13. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  14. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  15. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  16. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
  17. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  18. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  19. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 212
  20. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 206
  21. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  22. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 199
  23. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  24. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  25. 2019CvxNet: Learnable Convex Decompositioncited by 174
  26. 2019PQ-NET: A Generative Part Seq2Seq Network for 3D Shapescited by 119
  27. 2020ParSeNet: A Parametric Surface Fitting Network for 3D Point Cloudscited by 96
  28. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  29. 2023PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstructioncited by 84
  30. 2018Learning Shape Priors for Single-View 3D Completion and Reconstructioncited by 68
  31. 2023ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusioncited by 65
  32. 2018Learning to Reconstruct Shapes from Unseen Classescited by 60
  33. 2021Neural Marching Cubescited by 50
  34. 2021SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpaintingcited by 48
  35. 20213D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Imagecited by 45
  36. 2017SurfNet: Generating 3D shape surfaces using deep residual networkscited by 32
  37. 2023TOSS:High-quality Text-guided Novel View Synthesis from a Single Imagecited by 26
  38. 2023iNVS: Repurposing Diffusion Inpainters for Novel View Synthesiscited by 20

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 18; 2025: 8; 2026: 2.

28 papers.

Sort:

By area:

  1. 2024Efficient4D: Fast Dynamic 3D Object Generation from a Single-view Videocited by 44
  2. 2024GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generationcited by 36
  3. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  4. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  5. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 34
  6. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
  7. 2024Switch Diffusion Transformer: Synergizing Denoising Tasks with Sparse Mixture-of-Expertscited by 31
  8. 2025MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusioncited by 28
  9. 2024Denoising Task Difficulty-based Curriculum for Training Diffusion Modelscited by 27
  10. 2024MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusioncited by 22
  11. 2025SteerX: Creating Any Camera-Free 3D and 4D Scenes with Geometric Steeringcited by 15
  12. 2025VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting Generation with Flexible Pose and Multi-View Joint Modelingcited by 14
16 more
  1. 2024Flex3D: Feed-Forward 3D Generation with Flexible Reconstruction Model and Input View Curationcited by 12
  2. 2024LiftImage3D: Lifting Any Single Image to 3D Gaussians with Video Generation Priorscited by 12
  3. 2024Diffusion Model Patching via Mixture-of-Promptscited by 11
  4. 2025Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generatorcited by 11
  5. 2025ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encodingcited by 11
  6. 2025Zero-1-to-G: Taming Pretrained 2D Diffusion Model for Direct 3D Generationcited by 8
  7. 2026C2FG: Control Classifier-Free Guidance via Score Discrepancy Analysiscited by 7
  8. 2024GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splattingcited by 5
  9. 2024Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Datacited by 5
  10. 2024Jointly Generating Multi-view Consistent PBR Textures using Collaborative Controlcited by 4
  11. 2025Appreciate the View: A Task-Aware Evaluation Framework for Novel View Synthesiscited by 4
  12. 2024Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospectscited by 2
  13. 2024Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priorscited by 2
  14. 2024OneNet: A Channel-Wise 1D Convolutional U-Netcited by 0
  15. 2025ConsistentDreamer: View-Consistent Meshes Through Balanced Multi-View Gaussian Optimizationcited by 0
  16. 2026Opening the Black Box of Classifier-Free Guidance via Information Bottleneckcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Gen-3Diffusion: Realistic Image-to-3D Generation via 2D & 3D Diffusion Synergy
  2. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models
  3. 2025MEt3R: Measuring Multi-View Consistency in Generated Images
  4. 2025NOVA3D: Normal Aligned Video Diffusion Model for Single Image to 3D Generation
  5. 2025WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Image
  6. 2026Variance Reduction on the Camera Axis: Multi-View Score Distillation for 3D
  7. 2024Towards High-Fidelity 3D Portrait Generation with Rich Details by Cross-View Prior-Aware Diffusion
  8. 2024Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior