# Styleformer: Transformer based Generative Adversarial Networks with Style Vector

arXiv: https://arxiv.org/abs/2106.07023 · HTML: https://mlsanity.com/paper/2106.07023/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 70 references that are in the map.
- First posted on arXiv 13 Jun 2021.

## Builds on (most cited first, 30 of 48)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)

## Built on by (most cited first, 27 of 27)

- [Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theory](https://mlsanity.com/paper/2110.11291/) (2021)
- [Diffusion-GAN: Training GANs with Diffusion](https://mlsanity.com/paper/2206.02262/) (2022)
- [DiffiT: Diffusion Vision Transformers for Image Generation](https://mlsanity.com/paper/2312.02139/) (2023)
- [Reflected Diffusion Models](https://mlsanity.com/paper/2304.04740/) (2023)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Maximum Likelihood Training of Implicit Nonlinear Diffusion Models](https://mlsanity.com/paper/2205.13699/) (2022)
- [State-of-the-Art in the Architecture, Methods and Applications of StyleGAN](https://mlsanity.com/paper/2202.14020/) (2022)
- [ITTR: Unpaired Image-to-Image Translation with Transformers](https://mlsanity.com/paper/2203.16015/) (2022)
- [DomainStudio: Fine-Tuning Diffusion Models for Domain-Driven Image Generation using Limited Data](https://mlsanity.com/paper/2306.14153/) (2023)
- [PART: Pre-trained Authorship Representation Transformer](https://mlsanity.com/paper/2209.15373/) (2022)
- [Understanding writing style in social media with a supervised contrastively pre-trained transformer](https://mlsanity.com/paper/2310.11081/) (2023)
- [Transformer-based Generative Adversarial Networks in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2302.08641/) (2023)
- [Learning Disentangled Prompts for Compositional Image Synthesis](https://mlsanity.com/paper/2306.00763/) (2023)
- [Diffusion Models with Deterministic Normalizing Flow Priors](https://mlsanity.com/paper/2309.01274/) (2023)
- [A High-Quality Robust Diffusion Framework for Corrupted Dataset](https://mlsanity.com/paper/2311.17101/) (2023)
- [Model Synthesis for Zero-Shot Model Attribution](https://mlsanity.com/paper/2307.15977/) (2023)
- [The Nuts and Bolts of Adopting Transformer in GANs](https://mlsanity.com/paper/2110.13107/) (2021)
- [PointCAT: Cross-Attention Transformer for point cloud](https://mlsanity.com/paper/2304.03012/) (2023)
- [Efficient generative adversarial networks using linear additive-attention Transformers](https://mlsanity.com/paper/2401.09596/) (2024)
- [Solving Prior Distribution Mismatch in Diffusion Models via Optimal Transport](https://mlsanity.com/paper/2410.13431/) (2024)
- [GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/2111.03162/) (2021)
- [LouvreSAE: Sparse Autoencoders for Interpretable and Controllable Style Transfer](https://mlsanity.com/paper/2512.18930/) (2025)
- [3DTextureTransformer: Geometry Aware Texture Generation for Arbitrary Mesh Topology](https://mlsanity.com/paper/2403.04225/) (2024)
- [Hardware-Friendly Diffusion Models with Fixed-Size Reusable Structures for On-Device Image Generation](https://mlsanity.com/paper/2411.06119/) (2024)
- [SCFANet: Style Distribution Constraint Feature Alignment Network For Pathological Staining Translation](https://mlsanity.com/paper/2504.00490/) (2025)
- [Physically-Guided Optical Inversion Enable Non-Contact Side-Channel Attack on Isolated Screens](https://mlsanity.com/paper/2604.13419/) (2026)
- [ROMS-IMLE: A Minimalist Approach to Competitive Single-Step Generative Modelling](https://mlsanity.com/paper/2607.19332/) (2026)

## Abstract (arXiv)

We propose Styleformer, which is a style-based generator for GAN architecture, but a convolution-free transformer-based generator. In our paper, we explain how a transformer can generate high-quality images, overcoming the disadvantage that convolution operations are difficult to capture global features in an image. Furthermore, we change the demodulation of StyleGAN2 and modify the existing transformer structure (e.g., residual connection, layer normalization) to create a strong style-based generator with a convolution-free structure. We also make Styleformer lighter by applying Linformer, enabling Styleformer to generate higher resolution images and result in improvements in terms of speed and memory. We experiment with the low-resolution image dataset such as CIFAR-10, as well as the high-resolution image dataset like LSUN-church. Styleformer records FID 2.82 and IS 9.94 on CIFAR-10, a benchmark dataset, which is comparable performance to the current state-of-the-art and outperforms all GAN-based generative models, including StyleGAN2-ADA with fewer parameters on the unconditional setting. We also both achieve new state-of-the-art with FID 15.17, IS 11.01, and FID 3.66, respectively on STL-10 and CelebA. We release our code at https://github.com/Jeeseung-Park/Styleformer.