mlsanityData to 7 Oct 2026

SeMv-3D: Towards Concurrency of Semantic and Multi-view Consistency in General Text-to-3D Generation

arXiv 2410.07658 ↗First posted 10 Oct 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

General Text-to-3D (GT23D) generation is crucial for creating diverse 3D content across objects and scenes, yet it faces two key challenges: 1) ensuring semantic consistency between input text and generated 3D models, and 2) maintaining multi-view consistency across different perspectives within 3D. Existing approaches typically address only one of these challenges, often leading to suboptimal results in semantic fidelity and structural coherence. To overcome these limitations, we propose SeMv-3D, a novel framework that jointly enhances semantic alignment and multi-view consistency in GT23D generation. At its core, we introduce Triplane Prior Learning (TPL), which effectively learns triplane priors by capturing spatial correspondences across three orthogonal planes using a dedicated Orthogonal Attention mechanism, thereby ensuring geometric consistency across viewpoints. Additionally, we present Prior-based Semantic Aligning in Triplanes (SAT), which enables consistent any-view synthesis by leveraging attention-based feature alignment to reinforce the correspondence between textual semantics and triplane representations. Extensive experiments demonstrate that our method sets a new state-of-the-art in multi-view consistency, while maintaining competitive performance in semantic consistency compared to methods focused solely on semantic alignment. These results emphasize the remarkable ability of our approach to effectively balance and excel in both dimensions, establishing a new benchmark in the field.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 59 of its 81 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

59 of 81 references.

References · 59

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  10. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  11. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  12. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
38 more
  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  4. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  5. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  6. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  7. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  8. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  9. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  10. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  11. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  12. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  13. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  14. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  15. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  16. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  17. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  18. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  19. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  20. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  21. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  22. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  23. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  24. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  25. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  26. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  27. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  28. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  29. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  30. 2023Vox-E: Text-guided Voxel Editing of 3D Objectscited by 114
  31. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  32. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  33. 2023SKED: Sketch-guided Text-based 3D Editingcited by 72
  34. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  35. 2024Advances in 3D Generation: A Surveycited by 62
  36. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  37. 2023Control3D: Towards Controllable Text-to-3D Generationcited by 46
  38. 2023Instant3D: Instant Text-to-3D Generationcited by 45

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.