mlsanityData to 7 Oct 2026

CogView: Mastering Text-to-Image Generation via Transformers

arXiv 2105.13290 ↗First posted 26 May 2021cs.CV cs.LGcited by 691

Not on the map. Why?

Cited by 221 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding. We propose CogView, a 4-billion-parameter Transformer with VQ-VAE tokenizer to advance this problem. We also demonstrate the finetuning strategies for various downstream tasks, e.g. style learning, super-resolution, text-image ranking and fashion design, and methods to stabilize pretraining, e.g. eliminating NaN losses. CogView achieves the state-of-the-art FID on the blurred MS COCO dataset, outperforming previous GAN-based models and a recent similar work DALL-E.

Key facts. Cited by 691 papers in the map (data through 7 Oct 2026). Builds on 37 of its 58 references that are in the map. First posted on arXiv 26 May 2021.

Builds on

37 of 58 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2016Layer Normalizationcited by 5,579
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2013Auto-Encoding Variational Bayescited by 3,980
25 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  7. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  9. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  10. 2020On Layer Normalization in the Transformer Architecturecited by 906
  11. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  12. 2016Pixel Recurrent Neural Networkscited by 814
  13. 2018Image Transformercited by 770
  14. 2016Generative Adversarial Text to Image Synthesiscited by 735
  15. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  16. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  17. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  18. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  19. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  20. 2015Generating Images from Captions with Attentioncited by 135
  21. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  22. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  23. 2021M6: A Chinese Multimodal Pretrainercited by 91
  24. 2020Text-to-Image Generation Grounded by Fine-Grained User Attentioncited by 36
  25. 2021Controllable Generation from Pre-trained Language Models via Inverse Promptingcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 27; 2022: 107; 2023: 225; 2024: 167; 2025: 111; 2026: 54.

691 papers.

Sort:

By area:

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  8. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  9. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  10. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  11. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
38 more
  1. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  2. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  3. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  4. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  5. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  6. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  7. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  8. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  9. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  10. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  11. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  12. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  13. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  14. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  15. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  16. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  17. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  18. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  19. 2023Zero-shot Image-to-Image Translationcited by 503
  20. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  21. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  22. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  23. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 481
  24. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  25. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  26. 2022Blended Latent Diffusioncited by 430
  27. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  28. 2021A Survey of Transformerscited by 399
  29. 2022Multimodal Learning with Transformers: A Surveycited by 399
  30. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  31. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 358
  32. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  33. 2022MAGVIT: Masked Generative Video Transformercited by 341
  34. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  35. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  36. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  37. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  38. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312

50 most cited of 691 All 691 (JSON)