# Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling

arXiv: https://arxiv.org/abs/1812.01608 · HTML: https://mlsanity.com/paper/1812.01608/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2018.

## Built on by (most cited first, 30 of 74)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Generating Images with Sparse Representations](https://mlsanity.com/paper/2103.03841/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [SinSR: Diffusion-Based Image Super-Resolution in a Single Step](https://mlsanity.com/paper/2311.14760/) (2023)
- [Autoregressive Diffusion Models](https://mlsanity.com/paper/2110.02037/) (2021)
- [Generative Modelling With Inverse Heat Dissipation](https://mlsanity.com/paper/2206.13397/) (2022)
- [Street-View Image Generation from a Bird's-Eye View Layout](https://mlsanity.com/paper/2301.04634/) (2023)
- [Normalizing Flows are Capable Generative Models](https://mlsanity.com/paper/2412.06329/) (2024)
- [Colorization Transformer](https://mlsanity.com/paper/2102.04432/) (2021)
- [PixelSynth: Generating a 3D-Consistent Experience from a Single Image](https://mlsanity.com/paper/2108.05892/) (2021)
- [GIVT: Generative Infinite-Vocabulary Transformers](https://mlsanity.com/paper/2312.02116/) (2023)
- [Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes](https://mlsanity.com/paper/2111.12701/) (2021)
- [Subspace Diffusion Generative Models](https://mlsanity.com/paper/2205.01490/) (2022)
- [Combiner: Full Attention Transformer with Sparse Computation Cost](https://mlsanity.com/paper/2107.05768/) (2021)
- [Bayesian Flow Networks](https://mlsanity.com/paper/2308.07037/) (2023)
- [Predicting Video with VQVAE](https://mlsanity.com/paper/2103.01950/) (2021)
- [Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Image](https://mlsanity.com/paper/2203.09457/) (2022)
- [Adversarial Text-to-Image Synthesis: A Review](https://mlsanity.com/paper/2101.09983/) (2021)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Densely connected normalizing flows](https://mlsanity.com/paper/2106.04627/) (2021)

## Abstract (arXiv)

The unconditional generation of high fidelity images is a longstanding benchmark for testing the performance of image decoders. Autoregressive image models have been able to generate small images unconditionally, but the extension of these methods to large images where fidelity can be more readily assessed has remained an open problem. Among the major challenges are the capacity to encode the vast previous context and the sheer difficulty of learning a distribution that preserves both global semantic coherence and exactness of detail. To address the former challenge, we propose the Subscale Pixel Network (SPN), a conditional decoder architecture that generates an image as a sequence of sub-images of equal size. The SPN compactly captures image-wide spatial dependencies and requires a fraction of the memory and the computation required by other fully autoregressive models. To address the latter challenge, we propose to use Multidimensional Upscaling to grow an image in both size and depth via intermediate stages utilising distinct SPNs. We evaluate SPNs on the unconditional generation of CelebAHQ of size 256 and of ImageNet from size 32 to 256. We achieve state-of-the-art likelihood results in multiple settings, set up new benchmark results in previously unexplored settings and are able to generate very high fidelity large scale samples on the basis of both datasets.