mlsanityData to 7 Oct 2026

Zero-Shot Text-to-Image Generation

arXiv 2102.12092 ↗First posted 24 Feb 2021cs.CV cs.LGcited by 4,276

Not on the map. Why?

Cited by 1,077 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Text-to-image generation has traditionally focused on finding better modeling assumptions for training on a fixed dataset. These assumptions might involve complex architectures, auxiliary losses, or side information such as object part labels or segmentation masks supplied during training. We describe a simple approach for this task based on a transformer that autoregressively models the text and image tokens as a single stream of data. With sufficient data and scale, our approach is competitive with previous domain-specific models when evaluated in a zero-shot fashion.

Key facts. Cited by 4,276 papers in the map (data through 7 Oct 2026). Builds on 37 of its 64 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

37 of 64 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2016Improved Techniques for Training GANscited by 2,938
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  10. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
25 more
  1. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  2. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  3. 2017Mixed Precision Trainingcited by 1,135
  4. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  5. 2016Generative Adversarial Text to Image Synthesiscited by 735
  6. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  7. 2015Generating Sentences from a Continuous Spacecited by 607
  8. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  9. 2020Jukebox: A Generative Model for Musiccited by 535
  10. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  11. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  12. 2015YFCC100M: The New Data in Multimedia Researchcited by 234
  13. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
  14. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  15. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  16. 2020Understanding the Difficulty of Training Transformerscited by 210
  17. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  18. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
  19. 2015Generating Images from Captions with Attentioncited by 135
  20. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  21. 2016Learning What and Where to Drawcited by 105
  22. 2017Learning with Latent Languagecited by 75
  23. 2020X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformerscited by 58
  24. 2017Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networkscited by 43
  25. 2020Text-to-Image Generation Grounded by Fine-Grained User Attentioncited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 205; 2022: 571; 2023: 1158; 2024: 1178; 2025: 845; 2026: 319.

4,276 papers.

Sort:

By area:

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2023Segment Anythingcited by 8,158
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2022Generative Adversarial Networkscited by 5,603
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  4. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  6. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  7. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  8. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  9. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  10. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  11. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  12. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  13. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  14. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  15. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  16. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  17. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  18. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  19. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  20. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  21. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  22. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  23. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  24. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  25. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  26. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  27. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  28. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  29. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  30. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  31. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  32. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  33. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  34. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  35. 2021Multiscale Vision Transformerscited by 896
  36. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  37. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  38. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847

50 most cited of 4,276 All 4,276 (JSON)