mlsanityData to 7 Oct 2026

Controllable Text-to-Image Generation

arXiv 1909.07083 ↗First posted 16 Sep 2019cs.CV cs.CL cs.LGcited by 187

Not on the map. Why?

Cited by 52 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we propose a novel controllable text-to-image generative adversarial network (ControlGAN), which can effectively synthesise high-quality images and also control parts of the image generation according to natural language descriptions. To achieve this, we introduce a word-level spatial and channel-wise attention-driven generator that can disentangle different visual attributes, and allow the model to focus on generating and manipulating subregions corresponding to the most relevant words. Also, a word-level discriminator is proposed to provide fine-grained supervisory feedback by correlating words with image regions, facilitating training an effective generator which is able to manipulate specific visual attributes without affecting the generation of other content. Furthermore, perceptual loss is adopted to reduce the randomness involved in the image generation, and to encourage the generator to manipulate specific attributes required in the modified text. Extensive experiments on benchmark datasets demonstrate that our method outperforms existing state of the art, and is able to effectively manipulate synthetic images using natural language descriptions. Code is available at https://github.com/mrlibw/ControlGAN.

Key facts. Cited by 187 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 39; 2023: 58; 2024: 34; 2025: 22; 2026: 8.

187 papers.

Sort:

By area:

  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  3. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  4. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  5. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  6. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  7. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  8. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  9. 2021GAN Inversion: A Surveycited by 366
  10. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  11. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  12. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
38 more
  1. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 207
  2. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
  3. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 164
  4. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Promptscited by 161
  5. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 149
  6. 2023Rich Human Feedback for Text-to-Image Generationcited by 139
  7. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  8. 2023Human Motion Generation: A Surveycited by 95
  9. 2023GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesiscited by 84
  10. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  11. 2021Towards Flexible Blind JPEG Artifacts Removalcited by 76
  12. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  13. 2022Towards Implicit Text-Guided 3D Shape Generationcited by 68
  14. 2023Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generationcited by 66
  15. 2021Improving Text-to-Image Synthesis Using Contrastive Learningcited by 62
  16. 2022ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Modelcited by 62
  17. 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 61
  18. 2023Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generationcited by 60
  19. 2021Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigationcited by 53
  20. 2021Text to Image Generation with Semantic-Spatial Aware GANcited by 53
  21. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  22. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  23. 2023Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languagescited by 52
  24. 2021Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualizationcited by 51
  25. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 49
  26. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  27. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 47
  28. 2024Context-Guided Spatio-Temporal Video Groundingcited by 45
  29. 2022Discrete Contrastive Diffusion for Cross-Modal Music and Image Generationcited by 44
  30. 2022Lip to Speech Synthesis with Visual Context Attentional GANcited by 41
  31. 2023Highly Personalized Text Embedding for Image Manipulation by Stable Diffusioncited by 39
  32. 2023StyleAdapter: A Unified Stylized Image Generation Modelcited by 39
  33. 2021M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformerscited by 36
  34. 2023Improving Image Restoration through Removing Degradations in Textual Representationscited by 36
  35. 2024Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generationcited by 35
  36. 2022AnyFace: Free-style Text-to-Face Synthesis and Manipulationcited by 33
  37. 2022StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesiscited by 33
  38. 2023UniGS: Unified Representation for Image Generation and Segmentationcited by 29

50 most cited of 187 All 187 (JSON)