mlsanityData to 14 Oct 2026

Focus on Neighbors and Know the Whole: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creation

arXiv 2408.13149 ↗First posted 23 Aug 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Generating dense multiview images from text prompts is crucial for creating high-fidelity 3D assets. Nevertheless, existing methods struggle with space-view correspondences, resulting in sparse and low-quality outputs. In this paper, we introduce CoSER, a novel consistent dense Multiview Text-to-Image Generator for Text-to-3D, achieving both efficiency and quality by meticulously learning neighbor-view coherence and further alleviating ambiguity through the swift traversal of all views. For achieving neighbor-view consistency, each viewpoint densely interacts with adjacent viewpoints to perceive the global spatial structure, and aggregates information along motion paths explicitly defined by physical principles to refine details. To further enhance cross-view consistency and alleviate content drift, CoSER rapidly scan all views in spiral bidirectional manner to aware holistic information and then scores each point based on semantic material. Subsequently, we conduct weighted down-sampling along the spatial dimension based on scores, thereby facilitating prominent information fusion across all views with lightweight computation. Technically, the core module is built by integrating the attention mechanism with a selective state space model, exploiting the robust learning capabilities of the former and the low overhead of the latter. Extensive evaluation shows that CoSER is capable of producing dense, high-fidelity, content-consistent multiview images that can be flexibly integrated into various 3D generation models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 46 of its 50 references that are in the map. First posted on arXiv 23 Aug 2024.

Builds on

46 of 50 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  12. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
34 more
  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  3. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  4. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  5. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  6. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  7. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  8. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  9. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  10. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  11. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 328
  12. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  13. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
  14. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  15. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 257
  16. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  17. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  18. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 158
  19. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 155
  20. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3Dcited by 134
  21. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generationcited by 133
  22. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Modelscited by 123
  23. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 100
  24. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 89
  25. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  26. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 74
  27. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
  28. 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Modelscited by 58
  29. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  30. 2023GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generationcited by 30
  31. 2023AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillationcited by 28
  32. 2023Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priorscited by 24
  33. 2023ED-NeRF: Efficient Text-Guided Editing of 3D Scene with Latent Space NeRFcited by 20
  34. 2024Hash3D: Training-free Acceleration for 3D Generationcited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024JointDreamer: Ensuring Geometry Consistency and Text Congruence in Text-to-3D Generation via Joint Score Distillation
  2. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models
  3. 2023T3Bench: Benchmarking Current Progress in Text-to-3D Generation
  4. 2024Improving Viewpoint Consistency in 3D Generation via Structure Feature and CLIP Guidance
  5. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis
  6. 20243D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation
  7. 2026Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generation
  8. 2024GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation