# STAR: Scale-wise Text-conditioned AutoRegressive image generation

arXiv: https://arxiv.org/abs/2406.10797 · HTML: https://mlsanity.com/paper/2406.10797/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 44 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Builds on (most cited first, 30 of 34)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Scaling up GANs for Text-to-Image Synthesis](https://mlsanity.com/paper/2303.05511/) (2023)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation](https://mlsanity.com/paper/2403.04692/) (2024)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)

## Built on by (most cited first, 28 of 28)

- [HART: Efficient Visual Generation with Hybrid Autoregressive Transformer](https://mlsanity.com/paper/2410.10812/) (2024)
- [MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesis](https://mlsanity.com/paper/2407.07614/) (2024)
- [VAR-CLIP: Text-to-Image Generator with Visual Auto-Regressive Modeling](https://mlsanity.com/paper/2408.01181/) (2024)
- [Autoregressive Models in Vision: A Survey](https://mlsanity.com/paper/2411.05902/) (2024)
- [Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient](https://mlsanity.com/paper/2411.17787/) (2024)
- [CAR: Controllable Autoregressive Modeling for Visual Generation](https://mlsanity.com/paper/2410.04671/) (2024)
- [Customize Your Visual Autoregressive Recipe with Set Autoregressive Modeling](https://mlsanity.com/paper/2410.10511/) (2024)
- [Group Critical-token Policy Optimization for Autoregressive Image Generation](https://mlsanity.com/paper/2509.22485/) (2025)
- [X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Models](https://mlsanity.com/paper/2412.01824/) (2024)
- [Enhancing Reward Models for High-quality Image Generation: Beyond Text-Image Alignment](https://mlsanity.com/paper/2507.19002/) (2025)
- [Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspective](https://mlsanity.com/paper/2410.22217/) (2024)
- [Text-to-Image Synthesis: A Decade Survey](https://mlsanity.com/paper/2411.16164/) (2024)
- [DepthART: Monocular Depth Estimation as Autoregressive Refinement Task](https://mlsanity.com/paper/2409.15010/) (2024)
- [Marrying Autoregressive Transformer and Diffusion with Multi-Reference Autoregression](https://mlsanity.com/paper/2506.09482/) (2025)
- [Next-Scale Autoregressive Models for Text-to-Motion Generation](https://mlsanity.com/paper/2604.03799/) (2026)
- [GenAR: Next-Scale Autoregressive Generation for Spatial Gene Expression Prediction](https://mlsanity.com/paper/2510.04315/) (2025)
- [ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention](https://mlsanity.com/paper/2510.14882/) (2025)
- [AMS-KV: Adaptive KV Caching in Multi-Scale Visual Autoregressive Transformers](https://mlsanity.com/paper/2511.16047/) (2025)
- [VAR RL Done Right: Tackling Asynchronous Policy Conflicts in Visual Autoregressive Generation](https://mlsanity.com/paper/2601.02256/) (2026)
- [Alchemist: Unlocking Efficiency in Text-to-Image Model Training via Meta-Gradient Data Selection](https://mlsanity.com/paper/2512.16905/) (2025)
- [Beyond Patches: Global-aware Autoregressive Model for Multimodal Few-Shot Font Generation](https://mlsanity.com/paper/2601.01593/) (2026)
- [Adaptive Visual Autoregressive Acceleration via Dual-Linkage Entropy Analysis](https://mlsanity.com/paper/2602.01345/) (2026)
- [Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Models](https://mlsanity.com/paper/2604.14591/) (2026)
- [Hierarchical Vision-Language Alignment for Text-to-Image Generation via Diffusion Models](https://mlsanity.com/paper/2501.00917/) (2025)
- [Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstruction](https://mlsanity.com/paper/2510.22335/) (2025)
- [Progressive Checkerboards for Autoregressive Multiscale Image Generation](https://mlsanity.com/paper/2602.03811/) (2026)
- [Training-Free Semantic Correction for Autoregressive Visual Models](https://mlsanity.com/paper/2606.22550/) (2026)
- [MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations](https://mlsanity.com/paper/2606.23000/) (2026)

## Abstract (arXiv)

We introduce STAR, a text-to-image model that employs a scale-wise auto-regressive paradigm. Unlike VAR, which is constrained to class-conditioned synthesis for images up to 256×256, STAR enables text-driven image generation up to 1024×1024 through three key designs. First, we introduce a pre-trained text encoder to extract and adopt representations for textual constraints, enhancing details and generalizability. Second, given the inherent structural correlation across different scales, we leverage 2D Rotary Positional Encoding (RoPE) and tweak it into a normalized version, ensuring consistent interpretation of relative positions across token maps and stabilizing the training process. Third, we observe that simultaneously sampling all tokens within a single scale can disrupt inter-token relationships, leading to structural instability, particularly in high-resolution generation. To address this, we propose a novel stable sampling method that incorporates causal relationships into the sampling process, ensuring both rich details and stable structures. Compared to previous diffusion models and auto-regressive models, STAR surpasses existing benchmarks in fidelity, text-image consistency, and aesthetic quality, requiring just 2.21s for 1024×1024 images on A100. This highlights the potential of auto-regressive methods in high-quality image synthesis, offering new directions for the text-to-image generation.