# Recurrent Coupled Topic Modeling over Sequential Documents

arXiv: https://arxiv.org/abs/2106.13732 · HTML: https://mlsanity.com/paper/2106.13732/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 66 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency](https://mlsanity.com/paper/1611.01702/) (2016)
- [Dependent Multinomial Models Made Easy: Stick Breaking with the P'olya-Gamma Augmentation](https://mlsanity.com/paper/1506.05843/) (2015)
- [Dirichlet belief networks for topic structure learning](https://mlsanity.com/paper/1811.00717/) (2018)
- [Poisson-Randomized Gamma Dynamical Systems](https://mlsanity.com/paper/1910.12991/) (2019)
- [Topic Compositional Neural Language Model](https://mlsanity.com/paper/1712.09783/) (2017)
- [Document Informed Neural Autoregressive Topic Models with Distributional Prior](https://mlsanity.com/paper/1809.06709/) (2018)
- [Coupling Learning of Complex Interactions](https://mlsanity.com/paper/2007.13534/) (2020)
- [Poisson--Gamma Dynamical Systems](https://mlsanity.com/paper/1701.05573/) (2017)
- [Scalable Generalized Dynamic Topic Models](https://mlsanity.com/paper/1803.07868/) (2018)
- [Scaling up Dynamic Topic Models](https://mlsanity.com/paper/1602.06049/) (2016)
- [Topically Driven Neural Language Model](https://mlsanity.com/paper/1704.08012/) (2017)
- [Deep Poisson gamma dynamical systems](https://mlsanity.com/paper/1810.11209/) (2018)
- [Graph-Sparse LDA: A Topic Model with Structured Sparsity](https://mlsanity.com/paper/1410.4510/) (2014)
- [Nonparametric Bayesian Factor Analysis for Dynamic Count Matrices](https://mlsanity.com/paper/1512.08996/) (2015)
- [Efficient Correlated Topic Modeling with Topic Embedding](https://mlsanity.com/paper/1707.00206/) (2017)
- [Recurrent Dirichlet Belief Networks for Interpretable Dynamic Relational Data Modelling](https://mlsanity.com/paper/2002.10235/) (2020)
- [Deep Temporal-Recurrent-Replicated-Softmax for Topical Trends over Time](https://mlsanity.com/paper/1711.05626/) (2017)

## Abstract (arXiv)

The abundant sequential documents such as online archival, social media and news feeds are streamingly updated, where each chunk of documents is incorporated with smoothly evolving yet dependent topics. Such digital texts have attracted extensive research on dynamic topic modeling to infer hidden evolving topics and their temporal dependencies. However, most of the existing approaches focus on single-topic-thread evolution and ignore the fact that a current topic may be coupled with multiple relevant prior topics. In addition, these approaches also incur the intractable inference problem when inferring latent parameters, resulting in a high computational cost and performance degradation. In this work, we assume that a current topic evolves from all prior topics with corresponding coupling weights, forming the multi-topic-thread evolution. Our method models the dependencies between evolving topics and thoroughly encodes their complex multi-couplings across time steps. To conquer the intractable inference challenge, a new solution with a set of novel data augmentation techniques is proposed, which successfully discomposes the multi-couplings between evolving topics. A fully conjugate model is thus obtained to guarantee the effectiveness and efficiency of the inference technique. A novel Gibbs sampler with a backward-forward filter algorithm efficiently learns latent timeevolving parameters in a closed-form. In addition, the latent Indian Buffet Process (IBP) compound distribution is exploited to automatically infer the overall topic number and customize the sparse topic proportions for each sequential document without bias. The proposed method is evaluated on both synthetic and real-world datasets against the competitive baselines, demonstrating its superiority over the baselines in terms of the low per-word perplexity, high coherent topics, and better document time prediction.