# HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation

arXiv: https://arxiv.org/abs/2506.04421 · HTML: https://mlsanity.com/paper/2506.04421/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 44 of its 57 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 44)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://mlsanity.com/paper/2407.08608/) (2024)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)

## Built on by (most cited first, 20 of 20)

- [Visual Autoregressive Modeling for Instruction-Guided Image Editing](https://mlsanity.com/paper/2508.15772/) (2025)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation](https://mlsanity.com/paper/2604.11789/) (2026)
- [TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models](https://mlsanity.com/paper/2601.08011/) (2026)
- [ClusterMark: Towards Robust Watermarking for Autoregressive Image Generators with Visual Token Clustering](https://mlsanity.com/paper/2508.06656/) (2025)
- [Scale-Wise VAR is Secretly Discrete Diffusion](https://mlsanity.com/paper/2509.22636/) (2025)
- [Markovian Scale Prediction: A New Era of Visual Autoregressive Generation](https://mlsanity.com/paper/2511.23334/) (2025)
- [PRADA: Probability-Ratio-Based Attribution and Detection of Autoregressive-Generated Images](https://mlsanity.com/paper/2511.20068/) (2025)
- [Adaptive Visual Autoregressive Acceleration via Dual-Linkage Entropy Analysis](https://mlsanity.com/paper/2602.01345/) (2026)
- [SSG: Scaled Spatial Guidance for Multi-Scale Visual Autoregressive Generation](https://mlsanity.com/paper/2602.05534/) (2026)
- [UniCompress: Token Compression for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2603.11320/) (2026)
- [On the Robustness of Watermarking for Autoregressive Image Generation](https://mlsanity.com/paper/2604.11720/) (2026)
- [Collapse of Patches: Ranking Image Patches for Efficient Visual Modeling](https://mlsanity.com/paper/2511.22281/) (2025)
- [AlignVAR: Towards Globally Consistent Visual Autoregression for Image Super-Resolution](https://mlsanity.com/paper/2603.00589/) (2026)
- [Depth Adaptive Efficient Visual Autoregressive Modeling](https://mlsanity.com/paper/2604.17286/) (2026)
- [SplitAvatar: One-shot Head Avatar with Autoregressive Gaussian Splitting](https://mlsanity.com/paper/2605.25751/) (2026)
- [Edit the Bits, Diff the Codes: Bitwise Residual Editing for Visual Autoregressive Models](https://mlsanity.com/paper/2606.13558/) (2026)
- [Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modeling](https://mlsanity.com/paper/2609.11804/) (2026)
- [HALO: Enhancing Time Series Generation via Hyperspherical Latents and Masked AutoregRessive Modeling](https://mlsanity.com/paper/2609.34511/) (2026)
- [Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation](https://mlsanity.com/paper/2610.14162/) (2026)

## Abstract (arXiv)

Visual Auto-Regressive modeling (VAR) has shown promise in bridging the speed and quality gap between autoregressive image models and diffusion models. VAR reformulates autoregressive modeling by decomposing an image into successive resolution scales. During inference, an image is generated by predicting all the tokens in the next (higher-resolution) scale, conditioned on all tokens in all previous (lower-resolution) scales. However, this formulation suffers from reduced image quality due to the parallel generation of all tokens in a resolution scale; has sequence lengths scaling superlinearly in image resolution; and requires retraining to change the sampling schedule. We introduce Hierarchical Masked Auto-Regressive modeling (HMAR), a new image generation algorithm that alleviates these issues using next-scale prediction and masked prediction to generate high-quality images with fast sampling. HMAR reformulates next-scale prediction as a Markovian process, wherein the prediction of each resolution scale is conditioned only on tokens in its immediate predecessor instead of the tokens in all predecessor resolutions. When predicting a resolution scale, HMAR uses a controllable multi-step masked generation procedure to generate a subset of the tokens in each step. On ImageNet 256x256 and 512x512 benchmarks, HMAR models match or outperform parameter-matched VAR, diffusion, and autoregressive baselines. We develop efficient IO-aware block-sparse attention kernels that allow HMAR to achieve faster training and inference times over VAR by over 2.5x and 1.75x respectively, as well as over 3x lower inference memory footprint. Finally, HMAR yields additional flexibility over VAR; its sampling schedule can be changed without further training, and it can be applied to image editing tasks in a zero-shot manner.