# Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions

arXiv: https://arxiv.org/abs/1710.10304 · HTML: https://mlsanity.com/paper/1710.10304/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2017.

## Built on by (most cited first, 24 of 24)

- [Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation](https://mlsanity.com/paper/2106.03153/) (2021)
- [MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images](https://mlsanity.com/paper/2106.11944/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Modeling Image Composition for Complex Scene Generation](https://mlsanity.com/paper/2206.00923/) (2022)
- [Few-Shot Diffusion Models](https://mlsanity.com/paper/2205.15463/) (2022)
- [Learning a Prior over Intent via Meta-Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.12573/) (2018)
- [How to Train Your MAML to Excel in Few-Shot Classification](https://mlsanity.com/paper/2106.16245/) (2021)
- [FHIST: A Benchmark for Few-shot Classification of Histological Images](https://mlsanity.com/paper/2206.00092/) (2022)
- [Linguistically-Enriched and Context-Aware Zero-shot Slot Filling](https://mlsanity.com/paper/2101.06514/) (2021)
- [On Contrastive Representations of Stochastic Processes](https://mlsanity.com/paper/2106.10052/) (2021)
- [Learning Cross-Image Object Semantic Relation in Transformer for Few-Shot Fine-Grained Image Classification](https://mlsanity.com/paper/2207.00784/) (2022)
- [Neural Attention Models in Deep Learning: Survey and Taxonomy](https://mlsanity.com/paper/2112.05909/) (2021)
- [Few-Shot Semantic Parsing for New Predicates](https://mlsanity.com/paper/2101.10708/) (2021)
- [SCHA-VAE: Hierarchical Context Aggregation for Few-Shot Generation](https://mlsanity.com/paper/2110.12279/) (2021)
- [MetaCloth: Learning Unseen Tasks of Dense Fashion Landmark Detection from a Few Samples](https://mlsanity.com/paper/2112.02763/) (2021)
- [Few-shot Learning for Topic Modeling](https://mlsanity.com/paper/2104.09011/) (2021)
- [Adaptive Parametric Prototype Learning for Cross-Domain Few-Shot Classification](https://mlsanity.com/paper/2309.01342/) (2023)
- [Quasi-Bayesian Nonparametric Density Estimation via Autoregressive Predictive Updates](https://mlsanity.com/paper/2206.06462/) (2022)
- [Minimally Supervised Learning using Topological Projections in Self-Organizing Maps](https://mlsanity.com/paper/2401.06923/) (2024)
- [Meta-Learning for Neural Network-based Temporal Point Processes](https://mlsanity.com/paper/2401.15846/) (2024)
- [Meta-learning for Matrix Factorization without Shared Rows or Columns](https://mlsanity.com/paper/2106.15133/) (2021)
- [Meta-learning One-class Classifiers with Eigenvalue Solvers for Supervised Anomaly Detection](https://mlsanity.com/paper/2103.00684/) (2021)
- [DeltaGAN: Towards Diverse Few-shot Image Generation with Sample-Specific Delta](https://mlsanity.com/paper/2207.10271/) (2022)
- [Masked Auto-Regressive Variational Acceleration: Fast Inference Makes Practical Reinforcement Learning](https://mlsanity.com/paper/2511.15190/) (2025)

## Abstract (arXiv)

Deep autoregressive models have shown state-of-the-art performance in density estimation for natural images on large-scale datasets such as ImageNet. However, such models require many thousands of gradient-based weight updates and unique image examples for training. Ideally, the models would rapidly learn visual concepts from only a handful of examples, similar to the manner in which humans learns across many vision tasks. In this paper, we show how 1) neural attention and 2) meta learning techniques can be used in combination with autoregressive models to enable effective few-shot density estimation. Our proposed modifications to PixelCNN result in state-of-the art few-shot density estimation on the Omniglot dataset. Furthermore, we visualize the learned attention policy and find that it learns intuitive algorithms for simple tasks such as image mirroring on ImageNet and handwriting on Omniglot without supervision. Finally, we extend the model to natural images and demonstrate few-shot image generation on the Stanford Online Products dataset.