# TopicNet: Semantic Graph-Guided Topic Discovery

arXiv: https://arxiv.org/abs/2110.14286 · HTML: https://mlsanity.com/paper/2110.14286/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 27 Oct 2021.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Variational Lossy Autoencoder](https://mlsanity.com/paper/1611.02731/) (2016)
- [A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues](https://mlsanity.com/paper/1605.06069/) (2016)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Order-Embeddings of Images and Language](https://mlsanity.com/paper/1511.06361/) (2015)
- [Word Representations via Gaussian Embedding](https://mlsanity.com/paper/1412.6623/) (2014)
- [Towards Deeper Understanding of Variational Autoencoding Models](https://mlsanity.com/paper/1702.08658/) (2017)
- [Bayesian Attention Modules](https://mlsanity.com/paper/2010.10604/) (2020)
- [WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling](https://mlsanity.com/paper/1803.01328/) (2018)
- [Stochastic Backpropagation and Approximate Inference in Deep Generative Models](https://mlsanity.com/paper/1401.4082/) (2014)
- [Bayesian Attention Belief Networks](https://mlsanity.com/paper/2106.05251/) (2021)
- [Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network](https://mlsanity.com/paper/2107.02757/) (2021)
- [Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding](https://mlsanity.com/paper/2007.09536/) (2020)
- [Gamma Belief Networks](https://mlsanity.com/paper/1512.03081/) (2015)
- [Hierarchical Density Order Embeddings](https://mlsanity.com/paper/1804.09843/) (2018)
- [Dirichlet belief networks for topic structure learning](https://mlsanity.com/paper/1811.00717/) (2018)
- [The Poisson Gamma Belief Network](https://mlsanity.com/paper/1511.02199/) (2015)
- [Deep Autoencoding Topic Model with Scalable Hybrid Bayesian Inference](https://mlsanity.com/paper/2006.08804/) (2020)
- [Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC](https://mlsanity.com/paper/1706.01724/) (2017)
- [Deep Poisson gamma dynamical systems](https://mlsanity.com/paper/1810.11209/) (2018)
- [Convolutional Poisson Gamma Belief Network](https://mlsanity.com/paper/1905.05394/) (2019)

## Built on by (most cited first, 9 of 9)

- [Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings](https://mlsanity.com/paper/2203.01570/) (2022)
- [vONTSS: vMF based semi-supervised neural topic modeling with optimal transport](https://mlsanity.com/paper/2307.01226/) (2023)
- [HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding](https://mlsanity.com/paper/2210.10625/) (2022)
- [Knowledge-Aware Bayesian Deep Topic Model](https://mlsanity.com/paper/2209.14228/) (2022)
- [Self-supervised Topic Taxonomy Discovery in the Box Embedding Space](https://mlsanity.com/paper/2408.15050/) (2024)
- [Improving Sparse Autoencoder with Dynamic Attention](https://mlsanity.com/paper/2604.14925/) (2026)
- [HyHTM: Hyperbolic Geometry based Hierarchical Topic Models](https://mlsanity.com/paper/2305.09258/) (2023)
- [A Note on Bias to Complete](https://mlsanity.com/paper/2402.11710/) (2024)
- [Seeded Poisson Factorization: leveraging domain knowledge to fit topic models](https://mlsanity.com/paper/2503.02741/) (2025)

## Abstract (arXiv)

Existing deep hierarchical topic models are able to extract semantically meaningful topics from a text corpus in an unsupervised manner and automatically organize them into a topic hierarchy. However, it is unclear how to incorporate prior beliefs such as knowledge graph to guide the learning of the topic hierarchy. To address this issue, we introduce TopicNet as a deep hierarchical topic model that can inject prior structural knowledge as an inductive bias to influence learning. TopicNet represents each topic as a Gaussian-distributed embedding vector, projects the topics of all layers into a shared embedding space, and explores both the symmetric and asymmetric similarities between Gaussian embedding vectors to incorporate prior semantic hierarchies. With an auto-encoding variational inference network, the model parameters are optimized by minimizing the evidence lower bound and a regularization term via stochastic gradient descent. Experiments on widely used benchmarks show that TopicNet outperforms related deep topic models on discovering deeper interpretable topics and mining better document~representations.