# FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction

arXiv: https://arxiv.org/abs/2502.20313 · HTML: https://mlsanity.com/paper/2502.20313/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 53 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Builds on (most cited first, 30 of 48)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)
- [Show-o: One Single Transformer to Unify Multimodal Understanding and Generation](https://mlsanity.com/paper/2408.12528/) (2024)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets](https://mlsanity.com/paper/2202.00273/) (2022)
- [Scaling up GANs for Text-to-Image Synthesis](https://mlsanity.com/paper/2303.05511/) (2023)
- [Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks](https://mlsanity.com/paper/2206.08916/) (2022)
- [MAGVIT: Masked Generative Video Transformer](https://mlsanity.com/paper/2212.05199/) (2022)
- [Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis](https://mlsanity.com/paper/2412.04431/) (2024)

## Built on by (most cited first, 23 of 23)

- [Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities](https://mlsanity.com/paper/2505.02567/) (2025)
- [DetailFlow: 1D Coarse-to-Fine Autoregressive Image Generation via Next-Detail Prediction](https://mlsanity.com/paper/2505.21473/) (2025)
- [NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation](https://mlsanity.com/paper/2601.02204/) (2026)
- [FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge](https://mlsanity.com/paper/2505.14709/) (2025)
- [MVAR: Visual Autoregressive Modeling with Scale and Spatial Markovian Conditioning](https://mlsanity.com/paper/2505.12742/) (2025)
- [Stochastic Interpolants via Conditional Dependent Coupling](https://mlsanity.com/paper/2509.23122/) (2025)
- [Improving Progressive Generation with Decomposable Flow Matching](https://mlsanity.com/paper/2506.19839/) (2025)
- [Forecasting When to Forecast: Accelerating Diffusion Models with Confidence-Gated Taylor](https://mlsanity.com/paper/2508.02240/) (2025)
- [Diffusion-Free Graph Generation with Next-Scale Prediction](https://mlsanity.com/paper/2503.23612/) (2025)
- [Multi-Scale Local Speculative Decoding for Image Generation](https://mlsanity.com/paper/2601.05149/) (2026)
- [Exploiting Discriminative Codebook Prior for Autoregressive Image Generation](https://mlsanity.com/paper/2508.10719/) (2025)
- [Markovian Scale Prediction: A New Era of Visual Autoregressive Generation](https://mlsanity.com/paper/2511.23334/) (2025)
- [VAR RL Done Right: Tackling Asynchronous Policy Conflicts in Visual Autoregressive Generation](https://mlsanity.com/paper/2601.02256/) (2026)
- [LSRS: Latent Scale Rejection Sampling for Visual Autoregressive Modeling](https://mlsanity.com/paper/2512.03796/) (2025)
- [Inference-Time Scaling for Visual AutoRegressive modeling by Searching Representative Samples](https://mlsanity.com/paper/2601.07293/) (2026)
- [Adaptive Visual Autoregressive Acceleration via Dual-Linkage Entropy Analysis](https://mlsanity.com/paper/2602.01345/) (2026)
- [A Practical Investigation of Spatially-Controlled Image Generation with Transformers](https://mlsanity.com/paper/2507.15724/) (2025)
- [DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models](https://mlsanity.com/paper/2511.21415/) (2025)
- [Rethinking Training Dynamics in Scale-wise Autoregressive Generation](https://mlsanity.com/paper/2512.06421/) (2025)
- [HeatKV: Head-tuned KV-cache Compression for Visual Autoregressive Modeling](https://mlsanity.com/paper/2605.14877/) (2026)
- [Edit the Bits, Diff the Codes: Bitwise Residual Editing for Visual Autoregressive Models](https://mlsanity.com/paper/2606.13558/) (2026)
- [MEPA: Multi-Scale Representation Alignment for Visual Autoregressive Modeling with Mixture of Experts](https://mlsanity.com/paper/2607.00371/) (2026)
- [HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models](https://mlsanity.com/paper/2610.15297/) (2026)

## Abstract (arXiv)

This work challenges the residual prediction paradigm in visual autoregressive modeling and presents FlexVAR, a new Flexible Visual AutoRegressive image generation paradigm. FlexVAR facilitates autoregressive learning with ground-truth prediction, enabling each step to independently produce plausible images. This simple, intuitive approach swiftly learns visual distributions and makes the generation process more flexible and adaptable. Trained solely on low-resolution images (≤ 256px), FlexVAR can: (1) Generate images of various resolutions and aspect ratios, even exceeding the resolution of the training images. (2) Support various image-to-image tasks, including image refinement, in/out-painting, and image expansion. (3) Adapt to various autoregressive steps, allowing for faster inference with fewer steps or enhancing image quality with more steps. Our 1.0B model outperforms its VAR counterpart on the ImageNet 256×256 benchmark. Moreover, when zero-shot transfer the image generation process with 13 steps, the performance further improves to 2.08 FID, outperforming state-of-the-art autoregressive models AiM/VAR by 0.25/0.28 FID and popular diffusion models LDM/DiT by 1.52/0.19 FID, respectively. When transferring our 1.0B model to the ImageNet 512×512 benchmark in a zero-shot manner, FlexVAR achieves competitive results compared to the VAR 2.3B model, which is a fully supervised model trained at 512×512 resolution.