mlsanityData to 14 Oct 2026

Self-Attention Generative Adversarial Networks

arXiv 1805.08318 ↗First posted 21 May 2018stat.ML cs.LGcited by 809

Not on the map. Why?

Cited by 45 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we propose the Self-Attention Generative Adversarial Network (SAGAN) which allows attention-driven, long-range dependency modeling for image generation tasks. Traditional convolutional GANs generate high-resolution details as a function of only spatially local points in lower-resolution feature maps. In SAGAN, details can be generated using cues from all feature locations. Moreover, the discriminator can check that highly detailed features in distant portions of the image are consistent with each other. Furthermore, recent work has shown that generator conditioning affects GAN performance. Leveraging this insight, we apply spectral normalization to the GAN generator and find that this improves training dynamics. The proposed SAGAN achieves the state-of-the-art results, boosting the best published Inception score from 36.8 to 52.52 and reducing Frechet Inception distance from 27.62 to 18.65 on the challenging ImageNet dataset. Visualization of the attention layers shows that the generator leverages neighborhoods that correspond to object shapes rather than local regions of fixed shape.

Key facts. Cited by 809 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 2; 2020: 1; 2021: 316; 2022: 187; 2023: 141; 2024: 89; 2025: 52; 2026: 21.

809 papers.

Sort:

By area:

  1. 2022Generative Adversarial Networkscited by 5,627
  2. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  3. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  4. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  5. 2021Transformers in Vision: A Surveycited by 1,013
  6. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 959
  7. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  8. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  9. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  10. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  11. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  12. 2021GAN Inversion: A Surveycited by 367
38 more
  1. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 360
  2. 2022Detecting Deepfakes with Self-Blended Imagescited by 338
  3. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
  4. 2021Skillful Precipitation Nowcasting using Deep Generative Models of Radarcited by 311
  5. 2021VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalizationcited by 287
  6. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 264
  7. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  8. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
  9. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  10. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  11. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  12. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 227
  13. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
  14. 2022Visual Attention Networkcited by 210
  15. 2021Projected GANs Converge Fastercited by 208
  16. 2022Diffusion-GAN: Training GANs with Diffusioncited by 205
  17. 2023Data-centric Artificial Intelligence: A Surveycited by 203
  18. 2022Unsupervised Medical Image Translation with Adversarial Diffusion Modelscited by 192
  19. 2021ResViT: Residual vision transformers for multi-modal medical image synthesiscited by 177
  20. 2021The Values Encoded in Machine Learning Researchcited by 172
  21. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  22. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  23. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidancecited by 154
  24. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  25. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 151
  26. 2022Stripformer: Strip Transformer for Fast Image Deblurringcited by 140
  27. 2021Taming Visually Guided Sound Generationcited by 136
  28. 2023Hierarchical Fine-Grained Image Forgery Detection and Localizationcited by 134
  29. 2021Vision Transformer for Small-Size Datasetscited by 122
  30. 2021Instance-Conditioned GANcited by 121
  31. 2021Generative Adversarial Transformerscited by 120
  32. 2023Consistent View Synthesis with Pose-Guided Diffusion Modelscited by 117
  33. 2021Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localizationcited by 113
  34. 2021ViTGAN: Training GANs with Vision Transformerscited by 110
  35. 2021Beyond Self-attention: External Attention using Two Linear Layers for Visual Taskscited by 109
  36. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 109
  37. 2021LambdaNetworks: Modeling Long-Range Interactions Without Attentioncited by 107
  38. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 107

50 most cited of 809 All 809 (JSON)