mlsanityData to 14 Oct 2026

CorrespondentDream: Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences

arXiv 2404.10603 ↗First posted 16 Apr 2024cs.CVcited by 6

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Leveraging multi-view diffusion models as priors for 3D optimization have alleviated the problem of 3D consistency, e.g., the Janus face problem or the content drift problem, in zero-shot text-to-3D models. However, the 3D geometric fidelity of the output remains an unresolved issue; albeit the rendered 2D views are realistic, the underlying geometry may contain errors such as unreasonable concavities. In this work, we propose CorrespondentDream, an effective method to leverage annotation-free, cross-view correspondences yielded from the diffusion U-Net to provide additional 3D prior to the NeRF optimization process. We find that these correspondences are strongly consistent with human perception, and by adopting it in our loss design, we are able to produce NeRF models with geometries that are more coherent with common sense, e.g., more smoothed object surface, yielding higher 3D fidelity. We demonstrate the efficacy of our approach through various comparative qualitative results and a solid user study.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 33 of its 39 references that are in the map. First posted on arXiv 16 Apr 2024.

Builds on

33 of 39 references.

Closest ancestors · 12

Same direction, newest first

  1. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  2. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  3. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data13 Jun
  4. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  5. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  6. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Images29 Mar
  7. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  8. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  9. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  10. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  11. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  12. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 21

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  4. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  5. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 735
  6. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
  7. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  8. 2023Emergent Correspondence from Image Diffusioncited by 446
  9. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 310
  10. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 301
  11. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  12. 2021Self-Calibrating Neural Radiance Fieldscited by 204
9 more
  1. 2022SPARF: Neural Radiance Fields from Sparse and Noisy Posescited by 195
  2. 2018Neighbourhood Consensus Networkscited by 174
  3. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  4. 2021Learning Accurate Dense Correspondences and When to Trust Themcited by 113
  5. 2023NerfAcc: Efficient Sampling Accelerates NeRFscited by 90
  6. 2020Leveraging 2D Data to Learn Textured 3D Mesh Generationcited by 69
  7. 2023SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matchingcited by 38
  8. 2022TransforMatcher: Match-to-Match Attention for Semantic Correspondencecited by 29
  9. 2023Efficient Semantic Matching with Hypercolumn Correlationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025FreeAction: Training-Free Techniques for Enhanced Fidelity of Trajectory-to-Video Generationcited by 4
  2. 2026Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewardscited by 2
  3. 2025Similarity-Aware Selective State-Space Modeling for Semantic Correspondencecited by 1
  4. 2025Harnessing the Power of Training-Free Techniques in Text-to-2D Generation for Text-to-3D Generation via Score Distillation Samplingcited by 0
  5. 2025RapidMV: Leveraging Spatio-Angular Representations for Efficient and Consistent Text-to-Multi-View Synthesiscited by 0
  6. 2026Geometrically Consistent Multi-View Scene Generation from Freehand Sketchescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation
  2. 2024BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion
  3. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D
  4. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
  5. 2023GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generation
  6. 2023ControlDreamer: Blending Geometry and Style in Text-to-3D
  7. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors
  8. 20223DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models