# WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling

arXiv: https://arxiv.org/abs/1803.01328 · HTML: https://mlsanity.com/paper/1803.01328/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2018.

## Built on by (most cited first, 30 of 32)

- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings](https://mlsanity.com/paper/2203.01570/) (2022)
- [Bayesian Attention Belief Networks](https://mlsanity.com/paper/2106.05251/) (2021)
- [Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network](https://mlsanity.com/paper/2107.02757/) (2021)
- [A Survey on Neural Topic Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2401.15351/) (2024)
- [Neural Processes with Stochastic Attention: Paying more attention to the context dataset](https://mlsanity.com/paper/2204.05449/) (2022)
- [vONTSS: vMF based semi-supervised neural topic modeling with optimal transport](https://mlsanity.com/paper/2307.01226/) (2023)
- [HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding](https://mlsanity.com/paper/2210.10625/) (2022)
- [TopicNet: Semantic Graph-Guided Topic Discovery](https://mlsanity.com/paper/2110.14286/) (2021)
- [The future is different: Large pre-trained language models fail in prediction tasks](https://mlsanity.com/paper/2211.00384/) (2022)
- [Neural Dynamic Focused Topic Model](https://mlsanity.com/paper/2301.10988/) (2023)
- [Prototypes-oriented Transductive Few-shot Learning with Conditional Transport](https://mlsanity.com/paper/2308.03047/) (2023)
- [A Variational Edge Partition Model for Supervised Graph Representation Learning](https://mlsanity.com/paper/2202.03233/) (2022)
- [Knowledge-Aware Bayesian Deep Topic Model](https://mlsanity.com/paper/2209.14228/) (2022)
- [Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided Document Generation](https://mlsanity.com/paper/2312.11532/) (2023)
- [A Non-negative VAE:the Generalized Gamma Belief Network](https://mlsanity.com/paper/2408.03388/) (2024)
- [Bridging the Evaluation Gap: Leveraging Large Language Models for Topic Model Evaluation](https://mlsanity.com/paper/2502.07352/) (2025)
- [Sparse Autoencoders are Topic Models](https://mlsanity.com/paper/2511.16309/) (2025)
- [Matching Visual Features to Hierarchical Semantic Topics for Image Paragraph Captioning](https://mlsanity.com/paper/2105.04143/) (2021)
- [Beyond Spectral Decomposition: Bayesian Contrastive Learning and its Non-negative Formulation via Factor Analysis](https://mlsanity.com/paper/2407.21740/) (2024)
- [Enhancing Uncertainty Estimation and Interpretability via Bayesian Non-negative Decision Layer](https://mlsanity.com/paper/2505.22199/) (2025)
- [Toward Purpose-oriented Topic Model Evaluation enabled by Large Language Models](https://mlsanity.com/paper/2509.07142/) (2025)
- [Towards Generalising Neural Topical Representations](https://mlsanity.com/paper/2307.12564/) (2023)
- [Disentangled Generative Graph Representation Learning](https://mlsanity.com/paper/2408.13471/) (2024)
- [Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks](https://mlsanity.com/paper/2410.09696/) (2024)
- [Enhancing Topic Interpretability for Neural Topic Modeling through Topic-wise Contrastive Learning](https://mlsanity.com/paper/2412.17338/) (2024)
- [Towards Controllable Agent in MOBA Games with Generative Modeling](https://mlsanity.com/paper/2112.08093/) (2021)
- [Labeled Interactive Topic Models](https://mlsanity.com/paper/2311.09438/) (2023)
- [Multinomial belief networks for healthcare data](https://mlsanity.com/paper/2311.16909/) (2023)

## Abstract (arXiv)

To train an inference network jointly with a deep generative topic model, making it both scalable to big corpora and fast in out-of-sample prediction, we develop Weibull hybrid autoencoding inference (WHAI) for deep latent Dirichlet allocation, which infers posterior samples via a hybrid of stochastic-gradient MCMC and autoencoding variational Bayes. The generative network of WHAI has a hierarchy of gamma distributions, while the inference network of WHAI is a Weibull upward-downward variational autoencoder, which integrates a deterministic-upward deep neural network, and a stochastic-downward deep generative model based on a hierarchy of Weibull distributions. The Weibull distribution can be used to well approximate a gamma distribution with an analytic Kullback-Leibler divergence, and has a simple reparameterization via the uniform noise, which help efficiently compute the gradients of the evidence lower bound with respect to the parameters of the inference network. The effectiveness and efficiency of WHAI are illustrated with experiments on big corpora.