mlsanityData to 14 Oct 2026

Understand Before You Generate: Self-Guided Training for Autoregressive Image Generation

arXiv 2509.15185 ↗First posted 18 Sep 2025cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent studies have demonstrated the importance of high-quality visual representations in image generation and have highlighted the limitations of generative models in image understanding. As a generative paradigm originally designed for natural language, autoregressive models face similar challenges. In this work, we present the first systematic investigation into the mechanisms of applying the next-token prediction paradigm to the visual domain. We identify three key properties that hinder the learning of high-level visual semantics: local and conditional dependence, inter-step semantic inconsistency, and spatial invariance deficiency. We show that these issues can be effectively addressed by introducing self-supervised objectives during training, leading to a novel training framework, Self-guided Training for AutoRegressive models (ST-AR). Without relying on pre-trained representation models, ST-AR significantly enhances the image understanding ability of autoregressive models and leads to improved generation quality. Specifically, ST-AR brings approximately 42% FID improvement for LlamaGen-L and 49% FID improvement for LlamaGen-XL, while maintaining the same sampling strategy.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 53 of its 58 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

53 of 58 references.

References · 53

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  11. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  12. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
38 more
  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2022Generative Adversarial Networkscited by 5,627
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  9. 2023Qwen Technical Reportcited by 3,344
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  12. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  13. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  14. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  15. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  16. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  17. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  18. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  19. 2023PaLM 2 Technical Reportcited by 1,162
  20. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  21. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  22. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  23. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  24. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  25. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  26. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  27. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
  28. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  29. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  30. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  31. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  32. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  33. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  34. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 94
  35. 2023What Do Self-Supervised Vision Transformers Learn?cited by 93
  36. 2024FiT: Flexible Vision Transformer for Diffusion Modelcited by 85
  37. 2024Parallelized Autoregressive Visual Generationcited by 60
  38. 2025Transition Models: Rethinking the Generative Learning Objectivecited by 34

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025Autoregressive Image Generation Needs Only a Few Lines of Cached Tokenscited by 2
  2. 2026Mirai: Autoregressive Visual Generation Needs Foresightcited by 2
  3. 2026Unifying Contrastive and Generative Objectives for Visual Understanding and Text-to-Image Generationcited by 1
  4. 2026Adversarial Error Correction for Visual Autoregressive Generationcited by 1
  5. 2025FR-TTS: Test-Time Scaling for NTP-based Image Generation with Effective Filling-based Reward Signalcited by 0
  6. 2026Revisiting Autoregressive Models for Generative Image Classificationcited by 0
  7. 2026Efficient Training with Foresight: Multi-Token Auxiliary Supervision for Autoregressive Image Generationcited by 0