mlsanityData to 7 Oct 2026

Wavelets Are All You Need for Autoregressive Image Generation

arXiv 2406.19997 ↗First posted 28 Jun 2024cs.LG cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we take a new approach to autoregressive image generation that is based on two main ingredients. The first is wavelet image coding, which allows to tokenize the visual details of an image from coarse to fine details by ordering the information starting with the most significant bits of the most significant wavelet coefficients. The second is a variant of a language transformer whose architecture is re-designed and optimized for token sequences in this 'wavelet language'. The transformer learns the significant statistical correlations within a token sequence, which are the manifestations of well-known correlations between the wavelet subbands at various resolutions. We show experimental results with conditioning on the generation process.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

24 of 46 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
12 more
  1. 2018Hierarchical Neural Story Generationcited by 1,182
  2. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  3. 2020Efficient Transformers: A Surveycited by 784
  4. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  5. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
  6. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  7. 2020Focal Frequency Loss for Image Reconstruction and Synthesiscited by 204
  8. 2022Wavelet Diffusion Models are fast and scalable Image Generatorscited by 116
  9. 2022Formal Algorithms for Transformerscited by 70
  10. 2022Wavelet Score-Based Generative Modelingcited by 61
  11. 2021WaveFill: A Wavelet-based Generation Network for Image Inpaintingcited by 44
  12. 2024Compositional Text-to-Image Generation with Dense Blob Representationscited by 38

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 4.

9 papers.

  1. 2024Autoregressive Models in Vision: A Surveycited by 39
  2. 2024DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Spacecited by 32
  3. 2024Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenizationcited by 26
  4. 2024Spectral Image Tokenizercited by 15
  5. 2024Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspectivecited by 10
  6. 2025Flow Along the K-Amplitude for Generative Modelingcited by 3
  7. 2025WaRA: Wavelet Low-Rank Adaptation for Medical Image Classificationcited by 1
  8. 2025ActVAR: Activating Mixtures of Weights and Tokens for Efficient Visual Autoregressive Generationcited by 1
  9. 2025Nested AutoRegressive Modelscited by 0