mlsanityData to 14 Oct 2026

Switti: Designing Scale-Wise Transformers for Text-to-Image Synthesis

arXiv 2412.01819 ↗First posted 2 Dec 2024cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

This work presents Switti, a scale-wise transformer for text-to-image generation. We start by adapting an existing next-scale prediction autoregressive (AR) architecture to T2I generation, investigating and mitigating training stability issues in the process. Next, we argue that scale-wise transformers do not require causality and propose a non-causal counterpart facilitating ~21% faster sampling and lower memory usage while also achieving slightly better generation quality. Furthermore, we reveal that classifier-free guidance at high-resolution scales is often unnecessary and can even degrade performance. By disabling guidance at these scales, we achieve an additional sampling acceleration of ~32% and improve the generation of fine-grained details. Extensive human preference studies and automated evaluations show that Switti outperforms existing T2I AR models and competes with state-of-the-art T2I diffusion models while being up to 7x faster.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 66 of its 83 references that are in the map. First posted on arXiv 2 Dec 2024.

Builds on

66 of 83 references.

References · 66

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2024The Llama 3 Herd of Modelscited by 14,715
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  11. 2022Scalable Diffusion Models with Transformerscited by 6,054
  12. 2022Classifier-Free Diffusion Guidancecited by 5,847
38 more
  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  8. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  9. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  10. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  11. 2023Consistency Modelscited by 1,822
  12. 2020GLU Variants Improve Transformercited by 1,804
  13. 2019Root Mean Square Layer Normalizationcited by 1,480
  14. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  15. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  16. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  17. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  18. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  19. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  20. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  21. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  22. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  23. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  24. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  25. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  26. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  27. 2022On Distillation of Guided Diffusion Modelscited by 725
  28. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  29. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  30. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
  31. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  32. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  33. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  34. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  35. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  36. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  37. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 402
  38. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 256

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 16.

33 papers.

Sort:

By area:

  1. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 45
  2. 2025HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generationcited by 20
  3. 2025Scale-wise Distillation of Diffusion Modelscited by 11
  4. 2025Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editingcited by 7
  5. 2025CSD-VAR: Content-Style Decomposition in Visual Autoregressive Modelscited by 6
  6. 2025A Training-Free Style-aligned Image Generation with Scale-wise Autoregressive Modelcited by 4
  7. 2026TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Modelscited by 4
  8. 2026RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Spacecited by 4
  9. 2025bfD3QE: Learning Discrete Distribution Discrepancy-aware Quantization Error for Autoregressive-Generated Image Detectioncited by 3
  10. 2025CGCE: Classifier-Guided Concept Erasure in Generative Modelscited by 3
  11. 2026Multi-Scale Local Speculative Decoding for Image Generationcited by 3
  12. 2026ProGVC: Progressive-based Generative Video Compression via Auto-Regressive Context Modelingcited by 3
21 more
  1. 2025Scale-Wise VAR is Secretly Discrete Diffusioncited by 2
  2. 2025ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attentioncited by 2
  3. 2025Autoregressive Image Generation Needs Only a Few Lines of Cached Tokenscited by 2
  4. 2025Visual Autoregressive Modelling for Monocular Depth Estimationcited by 2
  5. 2026VAR RL Done Right: Tackling Asynchronous Policy Conflicts in Visual Autoregressive Generationcited by 2
  6. 2026Knowledge Distillation for Visual Autoregressive Modelscited by 2
  7. 2025Multi-scale Image Super Resolution with a Single Auto-Regressive Modelcited by 1
  8. 2025PRADA: Probability-Ratio-Based Attribution and Detection of Autoregressive-Generated Imagescited by 1
  9. 2026SSG: Scaled Spatial Guidance for Multi-Scale Visual Autoregressive Generationcited by 1
  10. 2026Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Modelscited by 1
  11. 2026HACK++: Towards More Effective Head-Aware Key-Value Compression for Efficient Visual Autoregressive Modelingcited by 1
  12. 2025A Training-Free Style-Personalization via SVD-Based Feature Decompositioncited by 0
  13. 2025Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstructioncited by 0
  14. 2025DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Modelscited by 0
  15. 2026StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Modelscited by 0
  16. 2026Revisiting Autoregressive Models for Generative Image Classificationcited by 0
  17. 2026Channel-wise Vector Quantizationcited by 0
  18. 2026Uni-AdaVD: Universal Concept Erasure for Visual Generation via Orthogonal Value Decompositioncited by 0
  19. 2026UniGen-AR: Unifying Visual Generation with Auto-Regressive Modelingcited by 0
  20. 2026SynVAR: Synergizing Spatial and Semantic Alignment in Visual Autoregressive Modelcited by 0
  21. 2026Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Scalability of Diffusion-based Text-to-Image Generation
  2. 2023KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis
  3. 2026Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders
  4. 2024Efficient Scaling of Diffusion Transformers for Text-to-Image Generation
  5. 2025Exploring the Deep Fusion of Large Language Models and Diffusion Transformers for Text-to-Image Synthesis
  6. 2026Unified Text-Image Generation with Weakness-Targeted Post-Training
  7. 2025DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture Design in Text to Image Generation
  8. 2024STAR: Scale-wise Text-conditioned AutoRegressive image generation