mlsanityData to 14 Oct 2026

Expressive Text-to-Image Generation with Rich Text

arXiv 2304.06720 ↗First posted 13 Apr 2023cs.CV cs.GR cs.LGcited by 89

Not on the map. Why?

Cited by 52 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Plain text has become a prevalent interface for text-to-image synthesis. However, its limited customization options hinder users from accurately describing desired outputs. For example, plain text makes it hard to specify continuous quantities, such as the precise RGB color value or importance of each word. Furthermore, creating detailed text prompts for complex scenes is tedious for humans to write and challenging for text encoders to interpret. To address these challenges, we propose using a rich-text editor supporting formats such as font style, size, color, and footnote. We extract each word's attributes from rich text to enable local style control, explicit token reweighting, precise color rendering, and detailed region synthesis. We achieve these capabilities through a region-based diffusion process. We first obtain each word's region based on attention maps of a diffusion process using plain text. For each region, we enforce its text attributes by creating region-specific detailed prompts and applying region-specific guidance, and maintain its fidelity against plain-text generation through region-based injections. We present various examples of image generation from rich text and demonstrate that our method outperforms strong baselines with quantitative evaluations.

Key facts. Cited by 89 papers in the map (data through 14 Oct 2026). Builds on 73 of its 110 references that are in the map. First posted on arXiv 13 Apr 2023.

Builds on

73 of 110 references.

References · 73

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2023Segment Anythingcited by 8,234
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2022Classifier-Free Diffusion Guidancecited by 5,847
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  4. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  5. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  6. 2022Video Diffusion Modelscited by 2,305
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  8. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  9. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  10. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  11. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  12. 2016Colorful Image Colorizationcited by 1,114
  13. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  14. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  15. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  16. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  17. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  18. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  19. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  20. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  21. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  22. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  23. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  24. 2023Zero-shot Image-to-Image Translationcited by 505
  25. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  26. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  27. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  28. 2023Universal Guidance for Diffusion Modelscited by 437
  29. 2022Blended Latent Diffusioncited by 432
  30. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  31. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
  32. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
  33. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
  34. 2023Pix2Video: Video Editing using Image Diffusioncited by 320
  35. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
  36. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  37. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
  38. 2022Novel View Synthesis with Diffusion Modelscited by 300

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 15; 2024: 48; 2025: 18; 2026: 8.

89 papers.

Sort:

By area:

  1. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 285
  2. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  3. 2023Grounded Text-to-Image Synthesis with Attention Refocusingcited by 164
  4. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 136
  5. 2024ReNoise: Real Image Inversion Through Iterative Noisingcited by 128
  6. 2024One-Step Image Translation with Text-to-Image Modelscited by 116
  7. 2024DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generationcited by 93
  8. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 91
  9. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  10. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalizationcited by 77
  11. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Modelscited by 63
  12. 2024Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Modelscited by 54
38 more
  1. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
  2. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  3. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  4. 2023Alchemist: Parametric Control of Material Properties with Diffusion Modelscited by 38
  5. 2024Compositional Text-to-Image Generation with Dense Blob Representationscited by 38
  6. 2024Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generationcited by 37
  7. 2023ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bankcited by 36
  8. 2024Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesiscited by 35
  9. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  10. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  11. 2023Prompt Highlighter: Interactive Control for Multi-Modal LLMscited by 29
  12. 2024It's All About Your Sketch: Democratising Sketch Control in Diffusion Modelscited by 29
  13. 2024Separable Multi-Concept Erasure from Diffusion Modelscited by 28
  14. 2024ColorPeel: Color Prompt Learning with Diffusion Models via Color and Shape Disentanglementcited by 28
  15. 2024Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformercited by 27
  16. 2024MC2: Multi-concept Guidance for Customized Multi-concept Generationcited by 26
  17. 2025Generating Multi-Image Synthetic Data for Text-to-Image Customizationcited by 25
  18. 2024ZeST: Zero-Shot Material Transfer from a Single Imagecited by 24
  19. 2024GrounDiT: Grounding Diffusion Transformers via Noisy Patch Transplantationcited by 24
  20. 2024MMTryon: Multi-Modal Multi-Reference Control for High-Quality Fashion Generationcited by 23
  21. 2023Text-Guided Synthesis of Eulerian Cinemagraphscited by 22
  22. 2023Mini-DALLE3: Interactive Text to Image by Prompting Large Language Modelscited by 22
  23. 2023Check, Locate, Rectify: A Training-Free Layout Calibration System for Text-to-Image Generationcited by 19
  24. 2024Consolidating Attention Features for Multi-view Image Editingcited by 19
  25. 2024ClassDiffusion: More Aligned Personalization Tuning with Explicit Class Guidancecited by 18
  26. 2024Bridging Different Language Models and Generative Vision Models for Text-to-Image Generationcited by 17
  27. 2024Self-Cross Diffusion Guidance for Text-to-Image Synthesis of Similar Subjectscited by 16
  28. 2024UniCtrl: Improving the Spatiotemporal Consistency of Text-to-Video Diffusion Models via Training-Free Unified Attention Controlcited by 14
  29. 2024Customizing Text-to-Image Diffusion with Object Viewpoint Controlcited by 13
  30. 2024Editable Image Elements for Controllable Synthesiscited by 13
  31. 2025DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generationcited by 13
  32. 2024GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Imagecited by 12
  33. 2024AWOL: Analysis WithOut synthesis using Languagecited by 12
  34. 20244-LEGS: 4D Language Embedded Gaussian Splattingcited by 12
  35. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  36. 2025Compass Control: Multi Object Orientation Control for Text-to-Image Generationcited by 11
  37. 2024AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generationcited by 10
  38. 2025Visually Guided Decoding: Gradient-Free Hard Prompt Inversion with Language Modelscited by 10

50 most cited of 89 All 89 (JSON)