mlsanityData to 14 Oct 2026

Resurrect Mask AutoRegressive Modeling for Efficient and Scalable Image Generation

arXiv 2507.13032 ↗First posted 17 Jul 2025cs.CVcited by 9

On the map: Efficient LLM inference and serving → Fast autoregressive visual generation

Abstract (arXiv)

AutoRegressive (AR) models have made notable progress in image generation, with Masked AutoRegressive (MAR) models gaining attention for their efficient parallel decoding. However, MAR models have traditionally underperformed when compared to standard AR models. This study refines the MAR architecture to improve image generation quality. We begin by evaluating various image tokenizers to identify the most effective one. Subsequently, we introduce an improved Bidirectional LLaMA architecture by replacing causal attention with bidirectional attention and incorporating 2D RoPE, which together form our advanced model, MaskGIL. Scaled from 111M to 1.4B parameters, MaskGIL achieves a FID score of 3.71, matching state-of-the-art AR models in the ImageNet 256x256 benchmark, while requiring only 8 inference steps compared to the 256 steps of AR models. Furthermore, we develop a text-driven MaskGIL model with 775M parameters for generating images from text at various resolutions. Beyond image generation, MaskGIL extends to accelerate AR-based generation and enable real-time speech-to-image conversion. Our codes and models are available at https://github.com/synbol/MaskGIL.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Fast autoregressive visual generation, one of 12 directions in Efficient LLM inference and serving. Builds on 45 of its 53 references that are in the map. First posted on arXiv 17 Jul 2025.

Builds on

45 of 53 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2023GPT-4 Technical Reportcited by 18,396
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2022Scalable Diffusion Models with Transformerscited by 6,054
  11. 2022Classifier-Free Diffusion Guidancecited by 5,847
  12. 2017Neural Discrete Representation Learningcited by 4,712
33 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  5. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  7. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  9. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  10. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  11. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  12. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  13. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  14. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  15. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  16. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  17. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  18. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  19. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  20. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 227
  21. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 155
  22. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 150
  23. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  24. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 132
  25. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 96
  26. 2023Stay on topic with Classifier-Free Guidancecited by 91
  27. 2022Improved Masked Image Generation with Token-Criticcited by 64
  28. 2025From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuningcited by 56
  29. 2024Towards Understanding the Working Mechanism of Text-to-Image Diffusion Modelcited by 48
  30. 2024Scalable Autoregressive Image Generation with Mambacited by 26
  31. 2023A Pytorch Reproduction of Masked Generative Image Transformercited by 25
  32. 2024Customize Your Visual Autoregressive Recipe with Set Autoregressive Modelingcited by 22
  33. 2023StraIT: Non-autoregressive Generation with Stratified Image Transformercited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 88
  2. 2025Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modelingcited by 37
  3. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  4. 2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Modelcited by 13
  5. 2025dMLLM-TTS: Self-Verified and Efficient Test-Time Scaling for Diffusion Multi-Modal Large Language Modelscited by 2
  6. 2025Low-Cost Test-Time Adaptation for Robust Video Editingcited by 1
  7. 2026GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editingcited by 1
  8. 2025Multi-Modal Machine Learning Framework for Predicting Early Recurrence of Brain Tumors Using MRI and Clinical Biomarkerscited by 0
  9. 2025A Multimodal Deep Learning Framework for Early Diagnosis of Liver Cancer via Optimized BiLSTM-AM-VMD Architecturecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer
  2. 2025LazyMAR: Accelerating Masked Autoregressive Models via Feature Caching
  3. 2025Fast Autoregressive Models for Continuous Latent Generation
  4. 2025Autoregressive Image Generation Needs Only a Few Lines of Cached Tokens
  5. 2025Hawk: Leveraging Spatial Context for Faster Autoregressive Text-to-Image Generation
  6. 2025Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy
  7. 2024MaskMamba: A Hybrid Mamba-Transformer Model for Masked Image Generation
  8. 2025Autoregressive Image Generation with Randomized Parallel Decoding