# Dynamic Topic Modeling with a Higher-Order Hypergraphical Representation

arXiv: https://arxiv.org/abs/2605.28269 · HTML: https://mlsanity.com/paper/2605.28269/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 16 of 16)

- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview](https://mlsanity.com/paper/1809.09573/) (2018)
- [Be More with Less: Hypergraph Attention Networks for Inductive Text Classification](https://mlsanity.com/paper/2011.00387/) (2020)
- [Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees](https://mlsanity.com/paper/1509.03025/) (2015)
- [Probabilistic Latent Semantic Analysis](https://mlsanity.com/paper/1301.6705/) (2013)
- [Community Detection for Hypergraph Networks via Regularized Tensor Power Iteration](https://mlsanity.com/paper/1909.06503/) (2019)
- [The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences](https://mlsanity.com/paper/1609.05820/) (2016)
- [Using SVD for Topic Modeling](https://mlsanity.com/paper/1704.07016/) (2017)
- [HyperBERT: Mixing Hypergraph-Aware Layers with Language Models for Node Classification on Text-Attributed Hypergraphs](https://mlsanity.com/paper/2402.07309/) (2024)
- [Learning representations of learning representations](https://mlsanity.com/paper/2404.08403/) (2024)
- [A General Latent Embedding Approach for Modeling Non-uniform High-dimensional Sparse Hypergraphs with Multiplicity](https://mlsanity.com/paper/2410.12108/) (2024)
- [Optimal Clustering of Discrete Mixtures: Binomial, Poisson, Block Models, and Multi-layer Networks](https://mlsanity.com/paper/2311.15598/) (2023)
- [Dynamic Topic Language Model on Heterogeneous Children's Mental Health Clinical Notes](https://mlsanity.com/paper/2312.14180/) (2023)
- [Multi-environment Topic Models](https://mlsanity.com/paper/2410.24126/) (2024)
- [Covariates-Adjusted Mixed-Membership Estimation: A Novel Network Model with Optimal Guarantees](https://mlsanity.com/paper/2502.06671/) (2025)

## Abstract (arXiv)

Dynamic topic modeling is widely used to analyze evolving trends in scientific literature, medical records, and social media. Traditional topic models represent each topic through a single probability vector on the multinomial simplex and implicitly couple word occurrence and repetition within one probabilistic mechanism. However, this formulation restricts the dependence structure among words and overlooks informative higher-order interactions, particularly in dynamic corpora with overlapping semantics. To address these limitations, we introduce a hypergraph representation of text where each document is modeled as a hyperedge connecting all co-occurring words, with repetition intensities encoded as node weights. This representation naturally separates word occurrence from repetition and induces a novel hypergraph-based multinomial distribution with a nonlinear normalization depending on the observed word set of each document. Building on this likelihood, we develop a dynamic topic modeling framework via structured low-rank factorizations with explicit temporal regularization on topic-word profiles. Moreover, we establish local convergence guarantees and derive non-asymptotic error bounds despite the intrinsic nonconvexity induced by bilinear factorization and document-specific nonlinear normalization. Numerical experiments on synthetic data and an application to the International Conference on Learning Representations (ICLR) corpus demonstrate consistent improvements over existing multinomial-based topic models.