# Wavelets Are All You Need for Autoregressive Image Generation

arXiv: https://arxiv.org/abs/2406.19997 · HTML: https://mlsanity.com/paper/2406.19997/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 28 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Images Speak in Images: A Generalist Painter for In-Context Visual Learning](https://mlsanity.com/paper/2212.02499/) (2022)
- [An Image is Worth 32 Tokens for Reconstruction and Generation](https://mlsanity.com/paper/2406.07550/) (2024)
- [Focal Frequency Loss for Image Reconstruction and Synthesis](https://mlsanity.com/paper/2012.12821/) (2020)
- [Wavelet Diffusion Models are fast and scalable Image Generators](https://mlsanity.com/paper/2211.16152/) (2022)
- [Formal Algorithms for Transformers](https://mlsanity.com/paper/2207.09238/) (2022)
- [Wavelet Score-Based Generative Modeling](https://mlsanity.com/paper/2208.05003/) (2022)
- [WaveFill: A Wavelet-based Generation Network for Image Inpainting](https://mlsanity.com/paper/2107.11027/) (2021)
- [Compositional Text-to-Image Generation with Dense Blob Representations](https://mlsanity.com/paper/2405.08246/) (2024)

## Built on by (most cited first, 9 of 9)

- [Autoregressive Models in Vision: A Survey](https://mlsanity.com/paper/2411.05902/) (2024)
- [DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Space](https://mlsanity.com/paper/2412.15032/) (2024)
- [Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization](https://mlsanity.com/paper/2412.05244/) (2024)
- [Spectral Image Tokenizer](https://mlsanity.com/paper/2412.09607/) (2024)
- [Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspective](https://mlsanity.com/paper/2410.22217/) (2024)
- [Flow Along the K-Amplitude for Generative Modeling](https://mlsanity.com/paper/2504.19353/) (2025)
- [WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification](https://mlsanity.com/paper/2506.24092/) (2025)
- [ActVAR: Activating Mixtures of Weights and Tokens for Efficient Visual Autoregressive Generation](https://mlsanity.com/paper/2511.12893/) (2025)
- [Nested AutoRegressive Models](https://mlsanity.com/paper/2510.23028/) (2025)

## Abstract (arXiv)

In this paper, we take a new approach to autoregressive image generation that is based on two main ingredients. The first is wavelet image coding, which allows to tokenize the visual details of an image from coarse to fine details by ordering the information starting with the most significant bits of the most significant wavelet coefficients. The second is a variant of a language transformer whose architecture is re-designed and optimized for token sequences in this 'wavelet language'. The transformer learns the significant statistical correlations within a token sequence, which are the manifestations of well-known correlations between the wavelet subbands at various resolutions. We show experimental results with conditioning on the generation process.