mlsanityData to 7 Oct 2026

An Image is Worth 32 Tokens for Reconstruction and Generation

arXiv 2406.07550 ↗First posted 11 Jun 2024cs.CVcited by 295

Not on the map. Why?

Cited by 61 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in generative models have highlighted the crucial role of image tokenization in the efficient synthesis of high-resolution images. Tokenization, which transforms images into latent representations, reduces computational demands compared to directly processing pixels and enhances the effectiveness and efficiency of the generation process. Prior methods, such as VQGAN, typically utilize 2D latent grids with fixed downsampling factors. However, these 2D tokenizations face challenges in managing the inherent redundancies present in images, where adjacent regions frequently display similarities. To overcome this issue, we introduce Transformer-based 1-Dimensional Tokenizer (TiTok), an innovative approach that tokenizes images into 1D latent sequences. TiTok provides a more compact latent representation, yielding substantially more efficient and effective representations than conventional techniques. For example, a 256 x 256 x 3 image can be reduced to just 32 discrete tokens, a significant reduction from the 256 or 1024 tokens obtained by prior methods. Despite its compact nature, TiTok achieves competitive performance to state-of-the-art approaches. Specifically, using the same generator framework, TiTok attains 1.97 gFID, outperforming MaskGIT baseline significantly by 4.21 at ImageNet 256 x 256 benchmark. The advantages of TiTok become even more significant when it comes to higher resolution. At ImageNet 512 x 512 benchmark, TiTok not only outperforms state-of-the-art diffusion model DiT-XL/2 (gFID 2.74 vs. 3.04), but also reduces the image tokens by 64x, leading to 410x faster generation process. Our best-performing variant can significantly surpasses DiT-XL/2 (gFID 2.13 vs. 3.04) while still generating high-quality samples 74x faster.

Key facts. Cited by 295 papers in the map (data through 7 Oct 2026). Builds on 64 of its 80 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

64 of 80 references.

References · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  10. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  11. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  12. 2023Visual Instruction Tuningcited by 7,792
38 more
  1. 2020End-to-End Object Detection with Transformerscited by 7,384
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  13. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  14. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  15. 2015VQA: Visual Question Answeringcited by 2,489
  16. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  17. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  18. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  19. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  20. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  21. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  22. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  23. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  24. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  25. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  26. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  27. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  28. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  29. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  30. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  31. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  32. 2022MAGVIT: Masked Generative Video Transformercited by 341
  33. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  34. 2020Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformerscited by 288
  35. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizercited by 260
  36. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  37. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
  38. 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 45; 2025: 131; 2026: 119.

295 papers.

Sort:

By area:

  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  2. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  3. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  4. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  5. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
  6. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 153
  7. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 148
  8. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
  9. 2024Randomized Autoregressive Visual Generationcited by 125
  10. 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generationcited by 112
  11. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 96
  12. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 93
38 more
  1. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  2. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 82
  3. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  4. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 77
  5. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  6. 2025Thinking with Generated Imagescited by 71
  7. 2025Dynamic Chunking for End-to-End Hierarchical Sequence Modelingcited by 65
  8. 2024Think While You Generate: Discrete Diffusion with Planned Denoisingcited by 61
  9. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  10. 2025GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generationcited by 55
  11. 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 55
  12. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusioncited by 53
  13. 2026Scaling Text-to-Image Diffusion Transformers with Representation Autoencoderscited by 49
  14. 2025Democratizing Text-to-Image Masked Generative Models with Compact Text-Aware One-Dimensional Tokenscited by 48
  15. 2024LARP: Tokenizing Videos with a Learned Autoregressive Generative Priorcited by 46
  16. 2024Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Headscited by 44
  17. 2024Scalable Image Tokenization with Index Backpropagation Quantizationcited by 44
  18. 2024Adaptive Length Image Tokenization via Recurrent Allocationcited by 40
  19. 2024Autoregressive Models in Vision: A Surveycited by 39
  20. 2024SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAEcited by 38
  21. 2026Driving on Registerscited by 38
  22. 2024Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models and Time-Dependent Layer Normalizationcited by 36
  23. 2024Language-Guided Image Tokenization for Generationcited by 36
  24. 2025Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenizationcited by 35
  25. 2025AlignTok: Aligning Visual Foundation Encoders to Tokenizers for Diffusion Modelscited by 35
  26. 2025Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communicationscited by 32
  27. 2025FlowTok: Flowing Seamlessly Across Text and Image Tokenscited by 31
  28. 2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 31
  29. 2025Halton Scheduler For Masked Generative Image Transformercited by 30
  30. 2025AToken: A Unified Tokenizer for Visioncited by 30
  31. 2025Bridging Continuous and Discrete Tokens for Autoregressive Visual Generationcited by 29
  32. 2024Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Surveycited by 27
  33. 2025Frequency Autoregressive Image Generation with Continuous Tokenscited by 27
  34. 2024Next Patch Prediction for Autoregressive Visual Generationcited by 25
  35. 2025"Principal Components" Enable A New Language of Imagescited by 25
  36. 2025SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generationcited by 21
  37. 2025FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenizationcited by 21
  38. 2026Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Modelscited by 21

50 most cited of 295 All 295 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025FlexTok: Resampling Images into 1D Token Sequences of Flexible Length
  2. 2025Learnings from Scaling Visual Tokenizers for Reconstruction and Generation
  3. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation
  4. 2022Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesis
  5. 2026Structured State-Space Regularization for Generation-Friendly Image Tokenization
  6. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes
  7. 2026What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization
  8. 2023Regularized Vector Quantization for Tokenized Image Synthesis