mlsanityData to 7 Oct 2026

Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

arXiv 2203.13131 ↗First posted 24 Mar 2022cs.CV cs.AI cs.CLcited by 487

Not on the map. Why?

Cited by 223 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent text-to-image generation methods provide a simple yet exciting conversion capability between text and image domains. While these methods have incrementally improved the generated image fidelity and text relevancy, several pivotal gaps remain unanswered, limiting applicability and quality. We propose a novel text-to-image method that addresses these gaps by (i) enabling a simple control mechanism complementary to text in the form of a scene, (ii) introducing elements that substantially improve the tokenization process by employing domain-specific knowledge over key image regions (faces and salient objects), and (iii) adapting classifier-free guidance for the transformer use case. Our model achieves state-of-the-art FID and human evaluation results, unlocking the ability to generate high fidelity images in a resolution of 512x512 pixels, significantly improving visual quality. Through scene controllability, we introduce several new capabilities: (i) Scene editing, (ii) text editing with anchor scenes, (iii) overcoming out-of-distribution text prompts, and (iv) story illustration generation, as demonstrated in the story we wrote.

Key facts. Cited by 487 papers in the map (data through 7 Oct 2026). Builds on 54 of its 73 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

54 of 73 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  11. 2017Neural Discrete Representation Learningcited by 4,650
  12. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  11. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  12. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  13. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  14. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  15. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  16. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  17. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  18. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  19. 2018Image Transformercited by 770
  20. 2016Generative Adversarial Text to Image Synthesiscited by 735
  21. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  22. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  23. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  24. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  25. 2017Unsupervised Image-to-Image Translation Networkscited by 602
  26. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  27. 2018Video-to-Video Synthesiscited by 388
  28. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  29. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  30. 2018A Note on the Inception Scorecited by 225
  31. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  32. 2019Controllable Text-to-Image Generationcited by 187
  33. 2019Self-Correction for Human Parsingcited by 179
  34. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 175
  35. 2015Generating Images from Captions with Attentioncited by 135
  36. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 105
  37. 2018Inferring Semantic Layout for Hierarchical Text-to-Image Synthesiscited by 101
  38. 2021Improving Text-to-Image Synthesis Using Contrastive Learningcited by 62

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 56; 2023: 201; 2024: 153; 2025: 61; 2026: 16.

487 papers.

Sort:

By area:

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  8. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  9. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  10. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  11. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  12. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
38 more
  1. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  2. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  3. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  4. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  5. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  6. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  7. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  8. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  9. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  10. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  11. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  12. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  13. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  14. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  15. 2023Zero-shot Image-to-Image Translationcited by 503
  16. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  17. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  18. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  19. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  20. 2022Blended Latent Diffusioncited by 430
  21. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  22. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  23. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  24. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  25. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  26. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  27. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  28. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  29. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  30. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  31. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  32. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  33. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  34. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  35. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  36. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  37. 2022AudioGen: Textually Guided Audio Generationcited by 292
  38. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 280

50 most cited of 487 All 487 (JSON)