# Hierarchical Autoregressive Image Models with Auxiliary Decoders

arXiv: https://arxiv.org/abs/1903.04933 · HTML: https://mlsanity.com/paper/1903.04933/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2019.

## Built on by (most cited first, 17 of 17)

- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis](https://mlsanity.com/paper/2108.08827/) (2021)
- [ShapeFormer: Transformer-based Shape Completion via Sparse Representation](https://mlsanity.com/paper/2201.10326/) (2022)
- [PoseGPT: Quantization-based 3D Human Motion Generation and Forecasting](https://mlsanity.com/paper/2210.10542/) (2022)
- [BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data](https://mlsanity.com/paper/2402.08093/) (2024)
- [Predicting Video with VQVAE](https://mlsanity.com/paper/2103.01950/) (2021)
- [Variable-rate discrete representation learning](https://mlsanity.com/paper/2103.06089/) (2021)
- [End-to-End Visual Editing with a Generatively Pre-Trained Artist](https://mlsanity.com/paper/2205.01668/) (2022)
- [Stable Attribute Group Editing for Reliable Few-shot Image Generation](https://mlsanity.com/paper/2302.00179/) (2023)
- [Cloning Deterministic Worlds: The Critical Role of Latent Geometry in Long-Horizon World Models](https://mlsanity.com/paper/2510.26782/) (2025)
- [Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps](https://mlsanity.com/paper/2501.06999/) (2025)
- [Text-Free Learning of a Natural Language Interface for Pretrained Face Generators](https://mlsanity.com/paper/2209.03953/) (2022)
- [Deep autoregressive modeling for land use land cover](https://mlsanity.com/paper/2401.01395/) (2024)
- [Boosting Predictive Performance on Tabular Data through Data Augmentation with Latent-Space Flow-Based Diffusion](https://mlsanity.com/paper/2511.16571/) (2025)
- [Adaptive Augmentation-Aware Latent Learning for Robust LiDAR Semantic Segmentation](https://mlsanity.com/paper/2603.01074/) (2026)

## Abstract (arXiv)

Autoregressive generative models of images tend to be biased towards capturing local structure, and as a result they often produce samples which are lacking in terms of large-scale coherence. To address this, we propose two methods to learn discrete representations of images which abstract away local detail. We show that autoregressive models conditioned on these representations can produce high-fidelity reconstructions of images, and that we can train autoregressive priors on these representations that produce samples with large-scale coherence. We can recursively apply the learning procedure, yielding a hierarchy of progressively more abstract image representations. We train hierarchical class-conditional autoregressive models on the ImageNet dataset and demonstrate that they are able to generate realistic images at resolutions of 128×128 and 256×256 pixels. We also perform a human evaluation study comparing our models with both adversarial and likelihood-based state-of-the-art generative models.