mlsanityData to 14 Oct 2026

PixelSNAIL: An Improved Autoregressive Generative Model

arXiv 1712.09763 ↗First posted 28 Dec 2017cs.LG stat.MLcited by 137

Not on the map. Why?

Cited by 31 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Autoregressive generative models consistently achieve the best results in density estimation tasks involving high dimensional data, such as images or audio. They pose density estimation as a sequence modeling task, where a recurrent neural network (RNN) models the conditional distribution over the next element conditioned on all previous elements. In this paradigm, the bottleneck is the extent to which the RNN can model long-range dependencies, and the most successful approaches rely on causal convolutions, which offer better access to earlier parts of the sequence than conventional RNNs. Taking inspiration from recent work in meta reinforcement learning, where dealing with long-range dependencies is also essential, we introduce a new generative model architecture that combines causal convolutions with self attention. In this note, we describe the resulting model and present state-of-the-art log-likelihood results on CIFAR-10 (2.85 bits per dim) and 32 × 32 ImageNet (3.80 bits per dim). Our implementation is available at https://github.com/neocxi/pixelsnail-public

Key facts. Cited by 137 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 Dec 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 26; 2022: 30; 2023: 23; 2024: 26; 2025: 24; 2026: 8.

137 papers.

Sort:

By area:

  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2022Video Diffusion Modelscited by 2,305
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  4. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 672
  5. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  6. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  7. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  8. 2021Score-based Generative Modeling in Latent Spacecited by 636
  9. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  10. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  11. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  12. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
38 more
  1. 2024Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokenscited by 147
  2. 2021Image-to-Image Translation: Methods and Applicationscited by 116
  3. 2021Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAEcited by 106
  4. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  5. 2021Understanding Failures in Out-of-Distribution Detection with Deep Generative Modelscited by 89
  6. 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
  7. 2022An Introduction to Neural Data Compressioncited by 79
  8. 2022PoseGPT: Quantization-based 3D Human Motion Generation and Forecastingcited by 78
  9. 2022SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantizationcited by 68
  10. 2024Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decodingcited by 66
  11. 2021Combiner: Full Attention Transformer with Sparse Computation Costcited by 64
  12. 2021Predicting Video with VQVAEcited by 61
  13. 2024Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesiscited by 56
  14. 2024Learning by Reconstruction Produces Uninformative Features For Perceptioncited by 55
  15. 2023Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEscited by 54
  16. 2021Densely connected normalizing flowscited by 49
  17. 2021Resampling Base Distributions of Normalizing Flowscited by 42
  18. 2021Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generationcited by 40
  19. 2024Autoregressive Models in Vision: A Surveycited by 39
  20. 2022Few-Shot Diffusion Modelscited by 38
  21. 2022A Study on the Evaluation of Generative Modelscited by 35
  22. 2023Autoregressive Conditional Neural Processescited by 33
  23. 2023Pixel-Inconsistency Modeling for Image Manipulation Localizationcited by 33
  24. 2022Full-band General Audio Synthesis with Score-based Diffusioncited by 32
  25. 2022Text2Human: Text-Driven Controllable Human Image Generationcited by 31
  26. 2022Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesiscited by 31
  27. 2025Continuous Diffusion Model for Language Modelingcited by 31
  28. 2025Neighboring Autoregressive Modeling for Efficient Visual Generationcited by 29
  29. 2023HQ-VAE: Hierarchical Discrete Representation Learning with Variational Bayescited by 27
  30. 2021Conditional Sound Generation Using Neural Discrete Time-Frequency Representation Learningcited by 24
  31. 2023Retrieval-Augmented Text-to-Audio Generationcited by 24
  32. 2022HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generatorcited by 22
  33. 2025AR-GRPO: Training Autoregressive Image Generation Models via Reinforcement Learningcited by 22
  34. 2022Few-shot Image Generation with Diffusion Modelscited by 21
  35. 2022Multi-scale Attention Flow for Probabilistic Time Series Forecastingcited by 19
  36. 2023Sequence Modeling with Multiresolution Convolutional Memorycited by 19
  37. 2024Continuous Speculative Decoding for Autoregressive Image Generationcited by 19
  38. 2023Deep Generative Models for Detector Signature Simulation: A Taxonomic Reviewcited by 18

50 most cited of 137 All 137 (JSON)