# Switti: Designing Scale-Wise Transformers for Text-to-Image Synthesis

arXiv: https://arxiv.org/abs/2412.01819 · HTML: https://mlsanity.com/paper/2412.01819/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 83 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 30 of 66)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)

## Built on by (most cited first, 30 of 33)

- [InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generation](https://mlsanity.com/paper/2511.04675/) (2025)
- [HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation](https://mlsanity.com/paper/2506.04421/) (2025)
- [Scale-wise Distillation of Diffusion Models](https://mlsanity.com/paper/2503.16397/) (2025)
- [Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editing](https://mlsanity.com/paper/2509.01984/) (2025)
- [CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models](https://mlsanity.com/paper/2507.13984/) (2025)
- [A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Model](https://mlsanity.com/paper/2504.06144/) (2025)
- [TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models](https://mlsanity.com/paper/2601.08011/) (2026)
- [RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space](https://mlsanity.com/paper/2602.20685/) (2026)
- [bfD3QE: Learning Discrete Distribution Discrepancy-aware Quantization Error for Autoregressive-Generated Image Detection](https://mlsanity.com/paper/2510.05891/) (2025)
- [CGCE: Classifier-Guided Concept Erasure in Generative Models](https://mlsanity.com/paper/2511.05865/) (2025)
- [Multi-Scale Local Speculative Decoding for Image Generation](https://mlsanity.com/paper/2601.05149/) (2026)
- [ProGVC: Progressive-based Generative Video Compression via Auto-Regressive Context Modeling](https://mlsanity.com/paper/2603.17546/) (2026)
- [Scale-Wise VAR is Secretly Discrete Diffusion](https://mlsanity.com/paper/2509.22636/) (2025)
- [ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention](https://mlsanity.com/paper/2510.14882/) (2025)
- [Autoregressive Image Generation Needs Only a Few Lines of Cached Tokens](https://mlsanity.com/paper/2512.04857/) (2025)
- [Visual Autoregressive Modelling for Monocular Depth Estimation](https://mlsanity.com/paper/2512.22653/) (2025)
- [VAR RL Done Right: Tackling Asynchronous Policy Conflicts in Visual Autoregressive Generation](https://mlsanity.com/paper/2601.02256/) (2026)
- [Knowledge Distillation for Visual Autoregressive Models](https://mlsanity.com/paper/2606.06078/) (2026)
- [Multi-scale Image Super Resolution with a Single Auto-Regressive Model](https://mlsanity.com/paper/2506.04990/) (2025)
- [PRADA: Probability-Ratio-Based Attribution and Detection of Autoregressive-Generated Images](https://mlsanity.com/paper/2511.20068/) (2025)
- [SSG: Scaled Spatial Guidance for Multi-Scale Visual Autoregressive Generation](https://mlsanity.com/paper/2602.05534/) (2026)
- [Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Models](https://mlsanity.com/paper/2604.14591/) (2026)
- [HACK++: Towards More Effective Head-Aware Key-Value Compression for Efficient Visual Autoregressive Modeling](https://mlsanity.com/paper/2606.08302/) (2026)
- [A Training-Free Style-Personalization via SVD-Based Feature Decomposition](https://mlsanity.com/paper/2507.04482/) (2025)
- [Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstruction](https://mlsanity.com/paper/2510.22335/) (2025)
- [DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models](https://mlsanity.com/paper/2511.21415/) (2025)
- [StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models](https://mlsanity.com/paper/2603.01757/) (2026)
- [Revisiting Autoregressive Models for Generative Image Classification](https://mlsanity.com/paper/2603.19122/) (2026)
- [Channel-wise Vector Quantization](https://mlsanity.com/paper/2605.26089/) (2026)
- [Uni-AdaVD: Universal Concept Erasure for Visual Generation via Orthogonal Value Decomposition](https://mlsanity.com/paper/2607.14521/) (2026)

## Abstract (arXiv)

This work presents Switti, a scale-wise transformer for text-to-image generation. We start by adapting an existing next-scale prediction autoregressive (AR) architecture to T2I generation, investigating and mitigating training stability issues in the process. Next, we argue that scale-wise transformers do not require causality and propose a non-causal counterpart facilitating ~21% faster sampling and lower memory usage while also achieving slightly better generation quality. Furthermore, we reveal that classifier-free guidance at high-resolution scales is often unnecessary and can even degrade performance. By disabling guidance at these scales, we achieve an additional sampling acceleration of ~32% and improve the generation of fine-grained details. Extensive human preference studies and automated evaluations show that Switti outperforms existing T2I AR models and competes with state-of-the-art T2I diffusion models while being up to 7x faster.