mlsanityData to 14 Oct 2026

ShapeWords: Guiding Text-to-Image Synthesis with 3D Shape-Aware Prompts

arXiv 2412.02912 ↗First posted 3 Dec 2024cs.CV cs.AI cs.GRcited by 2

Not on the map. Why?

Abstract (arXiv)

We introduce ShapeWords, an approach for synthesizing images based on 3D shape guidance and text prompts. ShapeWords incorporates target 3D shape information within specialized tokens embedded together with the input text, effectively blending 3D shape awareness with textual context to guide the image synthesis process. Unlike conventional shape guidance methods that rely on depth maps restricted to fixed viewpoints and often overlook full 3D structure or textual context, ShapeWords generates diverse yet consistent images that reflect both the target shape's geometry and the textual description. Experimental results show that ShapeWords produces images that are more text-compliant, aesthetically plausible, while also maintaining 3D shape awareness.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 45 of its 54 references that are in the map. First posted on arXiv 3 Dec 2024.

Builds on

45 of 54 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2020Denoising Diffusion Implicit Modelscited by 9,781
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  12. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
33 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  7. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  8. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  9. 2018Demystifying MMD GANscited by 1,293
  10. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  11. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  12. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  13. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  14. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  15. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
  16. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  17. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  18. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 231
  19. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  20. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 213
  21. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  22. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 200
  23. 2024Bilateral Reference for High-Resolution Dichotomous Image Segmentationcited by 112
  24. 20223DILG: Irregular Latent Grids for 3D Generative Modelingcited by 93
  25. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 91
  26. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generationcited by 81
  27. 20233D-aware Image Generation using 2D Diffusion Modelscited by 69
  28. 2023Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3Dcited by 59
  29. 2022A Study on the Evaluation of Generative Modelscited by 35
  30. 2024Ctrl-X: Controlling Structure and Appearance for Text-To-Image Generation Without Guidancecited by 34
  31. 2023CLiC: Concept Learning in Contextcited by 20
  32. 2024Learning Continuous 3D Words for Text-to-Image Generationcited by 16
  33. 2023The CLIP Model is Secretly an Image-to-Prompt Convertercited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Crafting Your Evolving Dreams: Concept-Incremental Versatile Customizationcited by 1
  2. 2026ScaleVid: Geometry-Aware Video Object Scaling with Mesh-Free Inferencecited by 0