mlsanityData to 7 Oct 2026

MaskBit: Embedding-free Image Generation via Bit Tokens

arXiv 2409.16211 ↗First posted 24 Sep 2024cs.CV cs.LGcited by 96

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Masked transformer models for class-conditional image generation have become a compelling alternative to diffusion models. Typically comprising two stages - an initial VQGAN model for transitioning between latent space and image space, and a subsequent Transformer model for image generation within latent space - these frameworks offer promising avenues for image synthesis. In this study, we present two primary contributions: Firstly, an empirical and systematic examination of VQGANs, leading to a modernized VQGAN. Secondly, a novel embedding-free generation network operating directly on bit tokens - a binary quantized representation of tokens with rich semantics. The first contribution furnishes a transparent, reproducible, and high-performing VQGAN model, enhancing accessibility and matching the performance of current state-of-the-art methods while revealing previously undisclosed details. The second contribution demonstrates that embedding-free image generation using bit tokens achieves a new state-of-the-art FID of 1.52 on the ImageNet 256x256 benchmark, with a compact generator model of mere 305M parameters. The code for this project is available on https://github.com/markweberdev/maskbit.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 50 of its 64 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

50 of 64 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models and Time-Dependent Layer Normalization13 Jun
  2. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  3. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  4. 2022Scalable Diffusion Models with Transformers19 Dec
  5. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  6. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning8 Aug
  7. 2022Classifier-Free Diffusion Guidance26 Jul
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
28 more
  1. 2016Improved Techniques for Training GANscited by 2,938
  2. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  3. 2018Group Normalizationcited by 1,514
  4. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  5. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  6. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  7. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  8. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  9. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  10. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  11. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  12. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  13. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  14. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  15. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  16. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  17. 2022MAGVIT: Masked Generative Video Transformercited by 341
  18. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  19. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  20. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 153
  21. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125
  22. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 105
  23. 2023GIVT: Generative Infinite-Vocabulary Transformerscited by 80
  24. 2022Improved Masked Image Generation with Token-Criticcited by 63
  25. 2023Online Clustered Codebookcited by 55
  26. 2022Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesiscited by 31
  27. 2022Draft-and-Revise: Effective Image Generation with Contextual RQ-Transformercited by 30
  28. 2023A Pytorch Reproduction of Masked Generative Image Transformercited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 52; 2026: 27.

96 papers.

Sort:

By area:

  1. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  2. 2024Randomized Autoregressive Visual Generationcited by 125
  3. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 93
  4. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  5. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 82
  6. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 77
  7. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  8. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  9. 2025GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generationcited by 55
  10. 2025Democratizing Text-to-Image Masked Generative Models with Compact Text-Aware One-Dimensional Tokenscited by 48
  11. 2024ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformercited by 45
  12. 2025Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenizationcited by 35
38 more
  1. 2025FlowTok: Flowing Seamlessly Across Text and Image Tokenscited by 31
  2. 2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 31
  3. 2024Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Surveycited by 27
  4. 2024Next Patch Prediction for Autoregressive Visual Generationcited by 25
  5. 2025"Principal Components" Enable A New Language of Imagescited by 25
  6. 2025Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminatorcited by 22
  7. 2025Autoregressive Image Generation with Randomized Parallel Decodingcited by 19
  8. 2025HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generationcited by 19
  9. 2025DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizercited by 13
  10. 2025ReDi: Rectified Discrete Flowcited by 13
  11. 2024XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generationcited by 12
  12. 2025MergeVQ: A Unified Framework for Visual Generation and Representation with Disentangled Token Merging and Quantizationcited by 12
  13. 2025TokBench: Evaluating Your Visual Tokenizer before Visual Generationcited by 12
  14. 2024Progressive Compositionality in Text-to-Image Generative Modelscited by 11
  15. 2024[MASK] is All You Needcited by 11
  16. 2026ELT: Elastic Looped Transformers for Visual Generationcited by 11
  17. 2024Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspectivecited by 10
  18. 2025Image Tokenizer Needs Post-Trainingcited by 10
  19. 2024A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Finegrained Image Generationcited by 9
  20. 2025Robust Latent Matters: Boosting Image Generation with Sampling Error Synthesiscited by 9
  21. 2025Continuous Visual Autoregressive Generation via Score Maximizationcited by 9
  22. 2026A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokenscited by 9
  23. 2025Deeply Supervised Flow-Based Generative Modelscited by 8
  24. 2025CODA: Repurposing Continuous VAEs for Discrete Tokenizationcited by 8
  25. 2025Locality-aware Parallel Decoding for Efficient Autoregressive Image Generationcited by 8
  26. 2026Autoregressive Image Generation with Masked Bit Modelingcited by 8
  27. 2024Efficient Generative Modeling with Residual Vector Quantization-Based Tokenscited by 7
  28. 2025Quantize-then-Rectify: Efficient VQ-VAE Trainingcited by 7
  29. 2025Discrete Tokenization for Multimodal LLMs: A Comprehensive Surveycited by 7
  30. 2025GPSToken: Gaussian Parameterized Spatially-adaptive Tokenization for Image Representation and Generationcited by 7
  31. 2024Preventing Local Pitfalls in Vector Quantization via Optimal Transportcited by 6
  32. 2025VTBench: Evaluating Visual Tokenizers for Autoregressive Image Generationcited by 6
  33. 2025Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesiscited by 6
  34. 2025REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularizationcited by 6
  35. 2025Spherical Leech Quantization for Visual Tokenization and Generationcited by 6
  36. 2025VA-π: Variational Policy Alignment for Pixel-Aware Autoregressive Generationcited by 6
  37. 2026Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Modelcited by 6
  38. 2026Frequency-Aware Flow Matching for High-Quality Image Generationcited by 6

50 most cited of 96 All 96 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesis
  2. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes
  3. 2024MaskMamba: A Hybrid Mamba-Transformer Model for Masked Image Generation
  4. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think
  5. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Training
  6. 2026FocusDiT: Masking Queries in Diffusion Transformers for Fine-grained Image Generation
  7. 2026A Hidden Semantic Bottleneck in Conditional Embeddings of Diffusion Transformers
  8. 2024Bag of Design Choices for Inference of High-Resolution Masked Generative Transformer