mlsanityData to 14 Oct 2026

ImageFolder: Autoregressive Image Generation with Folded Tokens

arXiv 2410.01756 ↗First posted 2 Oct 2024cs.CVcited by 94

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Image tokenizers are crucial for visual generative models, e.g., diffusion models (DMs) and autoregressive (AR) models, as they construct the latent representation for modeling. Increasing token length is a common approach to improve the image reconstruction quality. However, tokenizers with longer token lengths are not guaranteed to achieve better generation quality. There exists a trade-off between reconstruction and generation quality regarding token length. In this paper, we investigate the impact of token length on both image reconstruction and generation and provide a flexible solution to the tradeoff. We propose ImageFolder, a semantic tokenizer that provides spatially aligned image tokens that can be folded during autoregressive modeling to improve both generation efficiency and quality. To enhance the representative capability without increasing token length, we leverage dual-branch product quantization to capture different contexts of images. Specifically, semantic regularization is introduced in one branch to encourage compacted semantic information while another branch is designed to capture the remaining pixel-level details. Extensive experiments demonstrate the superior quality of image generation and shorter token length with ImageFolder tokenizer.

Key facts. Cited by 94 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 65 of its 78 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

65 of 78 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024MaskBit: Embedding-free Image Generation via Bit Tokens24 Sep
  2. 2022Scalable Diffusion Models with Transformers19 Dec
  3. 2022DiVAE: Photorealistic Images Synthesis with Denoising Diffusion Decoder1 Jun
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Score-based Generative Modeling in Latent Space10 Jun
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  8. 2020Denoising Diffusion Implicit Models6 Oct

Other references · 57

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  8. 2017Neural Discrete Representation Learningcited by 4,712
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  11. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
  1. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  3. 2016Improved Techniques for Training GANscited by 2,970
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  8. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  9. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  10. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  11. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  12. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  13. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  14. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  15. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  16. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  17. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  18. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  19. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  20. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  21. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  22. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  23. 2022MAGVIT: Masked Generative Video Transformercited by 346
  24. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
  25. 2022BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizerscited by 303
  26. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 302
  27. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  28. 2021PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformerscited by 204
  29. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 155
  30. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 150
  31. 2023Planting a SEED of Vision in Large Language Modelcited by 140
  32. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 136
  33. 20234M: Massively Multimodal Masked Modelingcited by 120
  34. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 107
  35. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 104
  36. 2023Return of Unconditional Generation: A Self-supervised Representation Generation Methodcited by 98
  37. 2024Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%cited by 68
  38. 2023Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantizationcited by 60

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 12; 2025: 44; 2026: 38.

94 papers.

Sort:

By area:

  1. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  2. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 202
  3. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 132
  4. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 96
  5. 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generationcited by 57
  6. 2025GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generationcited by 57
  7. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 56
  8. 2024ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformercited by 45
  9. 2025EVEv2: Improved Baselines for Encoder-Free Vision-Language Modelscited by 41
  10. 2024Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficientcited by 33
  11. 2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 33
  12. 2025AToken: A Unified Tokenizer for Visioncited by 30
38 more
  1. 2024CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive Predictioncited by 27
  2. 2025"Principal Components" Enable A New Language of Imagescited by 27
  3. 2024Next Patch Prediction for Autoregressive Visual Generationcited by 26
  4. 2025FlexVAR: Flexible Visual Autoregressive Modeling without Residual Predictioncited by 23
  5. 2025UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generationcited by 23
  6. 2025Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generationcited by 21
  7. 2025HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generationcited by 20
  8. 2025Latent Denoising Makes Good Tokenizerscited by 20
  9. 2025DetailFlow: 1D Coarse-to-Fine Autoregressive Image Generation via Next-Detail Predictioncited by 18
  10. 2025SCALAR: Scale-wise Controllable Visual Autoregressive Learningcited by 18
  11. 2025Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Modelscited by 17
  12. 2025VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Modelscited by 17
  13. 2025Hyperspherical Latents Improve Continuous-Token Autoregressive Generationcited by 16
  14. 2024X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Modelscited by 15
  15. 2025Frequency-Aware Autoregressive Modeling for Efficient High-Resolution Image Synthesiscited by 15
  16. 2025NFIG: Multi-Scale Autoregressive Image Generation via Frequency Orderingcited by 14
  17. 2024XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generationcited by 12
  18. 2024Visual Lexicon: Rich Image Features in Language Spacecited by 12
  19. 2025MergeVQ: A Unified Framework for Visual Generation and Representation with Disentangled Token Merging and Quantizationcited by 12
  20. 2025Image Tokenizer Needs Post-Trainingcited by 12
  21. 2025Robust Latent Matters: Boosting Image Generation with Sampling Error Synthesiscited by 10
  22. 2025D-AR: Diffusion via Autoregressive Modelscited by 9
  23. 2025WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstructioncited by 9
  24. 2026Condition Errors Refinement in Autoregressive Image Generation with Diffusion Losscited by 9
  25. 2026What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusioncited by 9
  26. 2025Make It Efficient: Dynamic Sparse Attention for Autoregressive Image Generationcited by 7
  27. 2025Quantize-then-Rectify: Efficient VQ-VAE Trainingcited by 7
  28. 2025Understand Before You Generate: Self-Guided Training for Autoregressive Image Generationcited by 7
  29. 2025DINO-Tok: Adapting DINO for Visual Tokenizerscited by 6
  30. 2026Improving Flexible Image Tokenizers for Autoregressive Image Generationcited by 6
  31. 2025OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactionscited by 5
  32. 2025Instella-T2I: Pushing the Limits of 1D Discrete Latent Space Image Generationcited by 5
  33. 2026ResTok: Learning Hierarchical Residuals in 1D Visual Tokenizers for Autoregressive Image Generationcited by 5
  34. 2026Fixed-Point Masked Generative Modelingcited by 5
  35. 2024Scaling Image Tokenizers with Grouped Spherical Quantizationcited by 4
  36. 2025Autoregressive Image Generation with Linear Complexity: A Spatial-Aware Decay Perspectivecited by 4
  37. 2026Boosting Latent Diffusion Models via Disentangled Representation Alignmentcited by 4
  38. 2025High-Fidelity Functional Ultrasound Reconstruction via A Visual Auto-Regressive Frameworkcited by 3

50 most cited of 94 All 94 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation
  2. 2025REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularization
  3. 2024Stabilize the Latent Space for Image Autoregressive Modeling: A Unified Perspective
  4. 2026GEAR: Guided End-to-End AutoRegression for Image Synthesis
  5. 2025Hita: Holistic Tokenizer for Autoregressive Image Generation
  6. 2025FlexTok: Resampling Images into 1D Token Sequences of Flexible Length
  7. 20243D representation in 512-Byte:Variational tokenizer is the key for autoregressive 3D generation
  8. 2025AlignTok: Aligning Visual Foundation Encoders to Tokenizers for Diffusion Models