mlsanityData to 7 Oct 2026

Efficient Image Generation with Variadic Attention Heads

arXiv 2211.05770 ↗First posted 10 Nov 2022cs.CV cs.AI cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

While the integration of transformers in vision models have yielded significant improvements on vision tasks they still require significant amounts of computation for both training and inference. Restricted attention mechanisms significantly reduce these computational burdens but come at the cost of losing either global or local coherence. We propose a simple, yet powerful method to reduce these trade-offs: allow the attention heads of a single transformer to attend to multiple receptive fields. We demonstrate our method utilizing Neighborhood Attention (NA) and integrate it into a StyleGAN based architecture for image generation. With this work, dubbed StyleNAT, we are able to achieve a FID of 2.05 on FFHQ, a 6% improvement over StyleGAN-XL, while utilizing 28% fewer parameters and with 4× the throughput capacity. StyleNAT achieves the Pareto Frontier on FFHQ-256 and demonstrates powerful and efficient image generation on other datasets. Our code and model checkpoints are publicly available at: https://github.com/SHI-Labs/StyleNAT

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 52 of its 80 references that are in the map. First posted on arXiv 10 Nov 2022.

Builds on

52 of 80 references.

References · 52

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2020Longformer: The Long-Document Transformercited by 3,362
  11. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  12. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
38 more
  1. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  2. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  3. 2018Demystifying MMD GANscited by 1,268
  4. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  5. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  6. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  7. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  8. 2021Video Swin Transformercited by 1,090
  9. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  10. 2018Self-Attention Generative Adversarial Networkscited by 809
  11. 2018Which Training Methods for GANs do actually Converge?cited by 672
  12. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  13. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  14. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  15. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 425
  16. 2016A Learned Representation For Artistic Stylecited by 340
  17. 2015A note on the evaluation of generative modelscited by 291
  18. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  19. 2022Neighborhood Attention Transformercited by 235
  20. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  21. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  22. 2021Projected GANs Converge Fastercited by 204
  23. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  24. 2019GANalyze: Toward Visual Definitions of Cognitive Image Propertiescited by 193
  25. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  26. 2021Generative Adversarial Transformerscited by 120
  27. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimationcited by 95
  28. 2021Real-time Neural Radiance Caching for Path Tracingcited by 91
  29. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71
  30. 2020Improved Consistency Regularization for GANscited by 67
  31. 2022Dilated Neighborhood Attention Transformercited by 64
  32. 2019HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Modelscited by 59
  33. 2021Improved Transformer for High-Resolution GANscited by 59
  34. 2021Data-Efficient Instance Generation from Instance Discriminationcited by 58
  35. 2019Approximation Capabilities of Neural ODEs and Invertible Residual Networkscited by 56
  36. 2019JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Videocited by 31
  37. 2021Compositional Transformers for Scene Generationcited by 24
  38. 2022A Tri-Layer Plugin to Improve Occluded Detectioncited by 21

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 4; 2025: 2; 2026: 2.

17 papers.

Sort:

By area:

  1. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 297
  2. 2023The Stable Signature: Rooting Watermarks in Latent Diffusion Modelscited by 289
  3. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Modelcited by 199
  4. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  5. 2023Zero-shot Generative Model Adaptation via Image-specific Prompt Learningcited by 27
  6. 2023Cousins Of The Vendi Score: A Family Of Similarity-Based Diversity Metrics For Science And Machine Learningcited by 24
  7. 2025Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Lightcited by 16
  8. 2024Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Levelcited by 14
  9. 2025One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Frameworkcited by 14
  10. 2024Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithmscited by 13
  11. 2023Single-Model Attribution of Generative Models Through Final-Layer Inversioncited by 10
  12. 2024EdgeNAT: Transformer for Efficient Edge Detectioncited by 6
5 more
  1. 2023Anomaly Score: Evaluating Generative Models and Individual Generated Images based on Complexity and Vulnerabilitycited by 2
  2. 2024Ultra-High-Resolution Image Synthesis with Pyramid Diffusion Modelcited by 1
  3. 2023NAC-TCN: Temporal Convolutional Networks with Causal Dilated Neighborhood Attention for Emotion Understandingcited by 0
  4. 2026GICDM: Mitigating Hubness for Reliable Distance-Based Generative Model Evaluationcited by 0
  5. 2026Improving Generative Adversarial Networks with Self-Distillationcited by 0