mlsanityData to 7 Oct 2026

Taming Transformers for High-Resolution Image Synthesis

arXiv 2012.09841 ↗First posted 17 Dec 2020cs.CVcited by 3,229

Not on the map. Why?

Cited by 956 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Designed to learn long-range interactions on sequential data, transformers continue to show state-of-the-art results on a wide variety of tasks. In contrast to CNNs, they contain no inductive bias that prioritizes local interactions. This makes them expressive, but also computationally infeasible for long sequences, such as high-resolution images. We demonstrate how combining the effectiveness of the inductive bias of CNNs with the expressivity of transformers enables them to model and thereby synthesize high-resolution images. We show how to (i) use CNNs to learn a context-rich vocabulary of image constituents, and in turn (ii) utilize transformers to efficiently model their composition within high-resolution images. Our approach is readily applied to conditional synthesis tasks, where both non-spatial information, such as object classes, and spatial information, such as segmentations, can control the generated image. In particular, we present the first results on semantically-guided synthesis of megapixel images with transformers and obtain the state of the art among autoregressive models on class-conditional ImageNet. Code and pretrained models can be found at https://github.com/CompVis/taming-transformers .

Key facts. Cited by 3,229 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 104; 2022: 321; 2023: 651; 2024: 760; 2025: 872; 2026: 521.

3,229 papers.

Sort:

By area:

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  9. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  10. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  12. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
38 more
  1. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  2. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  5. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  6. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  7. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  8. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  9. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  10. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  11. 2021Transformers in Vision: A Surveycited by 1,009
  12. 2022Diffusion Models in Vision: A Surveycited by 1,008
  13. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  14. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  15. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  16. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  17. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  18. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  19. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  20. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  21. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  22. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  23. 2023Adversarial Diffusion Distillationcited by 741
  24. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  25. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  26. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  27. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  28. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  29. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  30. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  31. 2023ModelScope Text-to-Video Technical Reportcited by 651
  32. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  33. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  34. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  35. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  36. 2021Score-based Generative Modeling in Latent Spacecited by 634
  37. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  38. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618

50 most cited of 3,229 All 3,229 (JSON)