# Few-shot Learning for Topic Modeling

arXiv: https://arxiv.org/abs/2104.09011 · HTML: https://mlsanity.com/paper/2104.09011/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 61 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Episodic Training for Domain Generalization](https://mlsanity.com/paper/1902.00113/) (2019)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Towards a Neural Statistician](https://mlsanity.com/paper/1606.02185/) (2016)
- [Compositional generalization through meta sequence-to-sequence learning](https://mlsanity.com/paper/1906.05381/) (2019)
- [Unsupervised Learning via Meta-Learning](https://mlsanity.com/paper/1810.02334/) (2018)
- [Hierarchically Structured Meta-learning](https://mlsanity.com/paper/1905.05301/) (2019)
- [Probabilistic Latent Semantic Analysis](https://mlsanity.com/paper/1301.6705/) (2013)
- [One-Shot Generalization in Deep Generative Models](https://mlsanity.com/paper/1603.05106/) (2016)
- [MetaFun: Meta-Learning with Iterative Functional Updates](https://mlsanity.com/paper/1912.02738/) (2019)
- [Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions](https://mlsanity.com/paper/1710.10304/) (2017)
- [Neural Topic Modeling with Continual Lifelong Learning](https://mlsanity.com/paper/2006.10909/) (2020)
- [The Variational Homoencoder: Learning to learn high capacity generative models from few examples](https://mlsanity.com/paper/1807.08919/) (2018)
- [Meta-Learning for Few-Shot Time Series Classification](https://mlsanity.com/paper/1909.07155/) (2019)
- [Variational Memory Addressing in Generative Models](https://mlsanity.com/paper/1709.07116/) (2017)
- [Meta Learning with Relational Information for Short Sequences](https://mlsanity.com/paper/1909.02105/) (2019)
- [Word Network Topic Model: A Simple but General Solution for Short and Imbalanced Texts](https://mlsanity.com/paper/1412.5404/) (2014)
- [Transfer Topic Modeling with Ease and Scalability](https://mlsanity.com/paper/1301.5686/) (2013)

## Built on by (most cited first, 5 of 5)

- [A Joint Learning Approach for Semi-supervised Neural Topic Modeling](https://mlsanity.com/paper/2204.03208/) (2022)
- [Human in the loop: How to effectively create coherent topics by manually labeling only a few documents per class](https://mlsanity.com/paper/2212.09422/) (2022)
- [Towards Generalising Neural Topical Representations](https://mlsanity.com/paper/2307.12564/) (2023)
- [FewTopNER: Integrating Few-Shot Learning with Topic Modeling and Named Entity Recognition in a Multilingual Framework](https://mlsanity.com/paper/2502.02391/) (2025)
- [Understanding Cross-Domain Adaptation in Low-Resource Topic Modeling](https://mlsanity.com/paper/2506.07453/) (2025)

## Abstract (arXiv)

Topic models have been successfully used for analyzing text documents. However, with existing topic models, many documents are required for training. In this paper, we propose a neural network-based few-shot learning method that can learn a topic model from just a few documents. The neural networks in our model take a small number of documents as inputs, and output topic model priors. The proposed method trains the neural networks such that the expected test likelihood is improved when topic model parameters are estimated by maximizing the posterior probability using the priors based on the EM algorithm. Since each step in the EM algorithm is differentiable, the proposed method can backpropagate the loss through the EM algorithm to train the neural networks. The expected test likelihood is maximized by a stochastic gradient descent method using a set of multiple text corpora with an episodic training framework. In our experiments, we demonstrate that the proposed method achieves better perplexity than existing methods using three real-world text document sets.