mlsanityData to 14 Oct 2026

Exploring Transformer Backbones for Image Diffusion Models

arXiv 2212.14678 ↗First posted 27 Dec 2022cs.CVcited by 3

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We present an end-to-end Transformer based Latent Diffusion model for image synthesis. On the ImageNet class conditioned generation task we show that a Transformer based Latent Diffusion model achieves a 14.1FID which is comparable to the 13.1FID score of a UNet based architecture. In addition to showing the application of Transformer models for Diffusion based image synthesis this simplification in architecture allows easy fusion and modeling of text and image data. The multi-head attention mechanism of Transformers enables simplified interaction between the image and text features which removes the requirement for crossattention mechanism in UNet based Diffusion models.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 15 of its 23 references that are in the map. First posted on arXiv 27 Dec 2022.

Builds on

15 of 23 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Classifier-Free Diffusion Guidance26 Jul
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  3. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 1.

3 papers.

  1. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  2. 2023Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines?cited by 10
  3. 2025Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applicationscited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2022Scalable Diffusion Models with Transformers
  2. 2024EDT: An Efficient Diffusion Transformer Framework Inspired by Human-like Sketching
  3. 2024LaVin-DiT: Large Vision Diffusion Transformer
  4. 2022Exploring Vision Transformers as Diffusion Learners
  5. 2025E-MMDiT: Revisiting Multimodal Diffusion Transformer Design for Fast Image Synthesis under Limited Resources
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
  7. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers
  8. 2025Diffusion Transformers with Representation Autoencoders