mlsanityData to 7 Oct 2026

Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generation

arXiv 2409.04410 ↗First posted 6 Sep 2024cs.CV cs.AIcited by 153

Not on the map. Why?

Cited by 24 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The Open-MAGVIT2 project produces an open-source replication of Google's MAGVIT-v2 tokenizer, a tokenizer with a super-large codebook (i.e., 218 codes), and achieves the state-of-the-art reconstruction performance on ImageNet and UCF benchmarks. We also provide a tokenizer pre-trained on large-scale data, significantly outperforming Cosmos on zero-shot benchmarks (1.93 vs. 0.78 rFID on ImageNet original resolution). Furthermore, we explore its application in plain auto-regressive models to validate scalability properties, producing a family of auto-regressive image generation models ranging from 300M to 1.5B. To assist auto-regressive models in predicting with a super-large vocabulary, we factorize it into two sub-vocabulary of different sizes by asymmetric token factorization, and further introduce ``next sub-token prediction'' to enhance sub-token interaction for better generation quality. We release all models and codes to foster innovation and creativity in the field of auto-regressive visual generation.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). Builds on 39 of its 53 references that are in the map. First posted on arXiv 6 Sep 2024.

Builds on

39 of 53 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2022Scalable Diffusion Models with Transformerscited by 5,931
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
27 more
  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  8. 2020GLU Variants Improve Transformercited by 1,724
  9. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  10. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  11. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  12. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  13. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  14. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  15. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  16. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  17. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  18. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  19. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  20. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  21. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 361
  22. 2022MAGVIT: Masked Generative Video Transformercited by 341
  23. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  24. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  25. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  26. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generationcited by 117
  27. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 105

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 25; 2025: 83; 2026: 45.

153 papers.

Sort:

By area:

  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  2. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  3. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 141
  4. 2024Randomized Autoregressive Visual Generationcited by 125
  5. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  6. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 96
  7. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  8. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 93
  9. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 83
  10. 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 83
  11. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 82
  12. 2025Harmonizing Visual Representations for Unified Multimodal Understanding and Generationcited by 74
38 more
  1. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  2. 2025VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizerscited by 67
  3. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  4. 2025GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generationcited by 55
  5. 2025TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generationcited by 50
  6. 2024GenXD: Generating Any 3D and 4D Scenescited by 49
  7. 2025UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encodingcited by 49
  8. 2024ControlAR: Controllable Image Generation with Autoregressive Modelscited by 48
  9. 2025Democratizing Text-to-Image Masked Generative Models with Compact Text-Aware One-Dimensional Tokenscited by 48
  10. 2025Unified Multimodal Discrete Diffusioncited by 46
  11. 2024Scalable Image Tokenization with Index Backpropagation Quantizationcited by 44
  12. 2025DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabulariescited by 42
  13. 2024Autoregressive Models in Vision: A Surveycited by 39
  14. 2025Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenizationcited by 35
  15. 2024Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficientcited by 32
  16. 2024Causal Diffusion Transformers for Generative Modelingcited by 31
  17. 2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 31
  18. 2025AToken: A Unified Tokenizer for Visioncited by 30
  19. 2024CAR: Controllable Autoregressive Modeling for Visual Generationcited by 29
  20. 2025Bridging Continuous and Discrete Tokens for Autoregressive Visual Generationcited by 29
  21. 2025Frequency Autoregressive Image Generation with Continuous Tokenscited by 27
  22. 2024Next Patch Prediction for Autoregressive Visual Generationcited by 25
  23. 2025UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generationcited by 23
  24. 2025FlexVAR: Flexible Visual Autoregressive Modeling without Residual Predictioncited by 22
  25. 2025Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminatorcited by 22
  26. 2025Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compressioncited by 22
  27. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  28. 2024Customize Your Visual Autoregressive Recipe with Set Autoregressive Modelingcited by 21
  29. 2025SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generationcited by 21
  30. 2025LightGen: Efficient Image Generation through Knowledge Distillation and Direct Preference Optimizationcited by 20
  31. 2025Towards Sequence Modeling Alignment between Tokenizer and Autoregressive Modelcited by 20
  32. 2025Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generationcited by 20
  33. 2025Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Modelscited by 18
  34. 2025SCALAR: Scale-wise Controllable Visual Autoregressive Learningcited by 18
  35. 2025DetailFlow: 1D Coarse-to-Fine Autoregressive Image Generation via Next-Detail Predictioncited by 17
  36. 2024X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Modelscited by 15
  37. 2024ACT-Bench: Towards Action Controllable World Models for Autonomous Drivingcited by 14
  38. 2024When Worse is Better: Navigating the compression-generation tradeoff in visual tokenizationcited by 14

50 most cited of 153 All 153 (JSON)