# Neural Models for Documents with Metadata

arXiv: https://arxiv.org/abs/1705.09296 · HTML: https://mlsanity.com/paper/1705.09296/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2017.

## Built on by (most cited first, 30 of 46)

- [DEMix Layers: Disentangling Domains for Modular Language Modeling](https://mlsanity.com/paper/2108.05036/) (2021)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Latent Diffusion Energy-Based Model for Interpretable Text Modeling](https://mlsanity.com/paper/2206.05895/) (2022)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning](https://mlsanity.com/paper/2211.12878/) (2022)
- [Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings](https://mlsanity.com/paper/2203.01570/) (2022)
- [Towards the TopMost: A Topic Modeling System Toolkit](https://mlsanity.com/paper/2309.06908/) (2023)
- [On the Affinity, Rationality, and Diversity of Hierarchical Topic Modeling](https://mlsanity.com/paper/2401.14113/) (2024)
- [A Survey on Neural Topic Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2401.15351/) (2024)
- [Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classification](https://mlsanity.com/paper/2108.11556/) (2021)
- [Topic Modeling as Multi-Objective Contrastive Optimization](https://mlsanity.com/paper/2402.07577/) (2024)
- [Are Neural Topic Models Broken?](https://mlsanity.com/paper/2210.16162/) (2022)
- [LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Models](https://mlsanity.com/paper/2406.09008/) (2024)
- [Challenges and Opportunities in Information Manipulation Detection: An Examination of Wartime Russian Media](https://mlsanity.com/paper/2205.12382/) (2022)
- [Neural Topic Modeling with Large Language Models in the Loop](https://mlsanity.com/paper/2411.08534/) (2024)
- [Neural Attention-Aware Hierarchical Topic Model](https://mlsanity.com/paper/2110.07161/) (2021)
- [Hierarchical Interpretation of Neural Text Classification](https://mlsanity.com/paper/2202.09792/) (2022)
- [Diversity-Aware Coherence Loss for Improving Neural Topic Models](https://mlsanity.com/paper/2305.16199/) (2023)
- [Bayesian Topic Regression for Causal Inference](https://mlsanity.com/paper/2109.05317/) (2021)
- [Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Media](https://mlsanity.com/paper/2205.03296/) (2022)
- [Bridging the Evaluation Gap: Leveraging Large Language Models for Topic Model Evaluation](https://mlsanity.com/paper/2502.07352/) (2025)
- [CUE: An Uncertainty Interpretation Framework for Text Classifiers Built on Pre-Trained Language Models](https://mlsanity.com/paper/2306.03598/) (2023)
- [Sparse Autoencoders are Topic Models](https://mlsanity.com/paper/2511.16309/) (2025)
- [Toward Purpose-oriented Topic Model Evaluation enabled by Large Language Models](https://mlsanity.com/paper/2509.07142/) (2025)
- [Adversarial Learning of Poisson Factorisation Model for Gauging Brand Sentiment in User Reviews](https://mlsanity.com/paper/2101.10150/) (2021)
- [Neural Topic Modeling with Deep Mutual Information Estimation](https://mlsanity.com/paper/2203.06298/) (2022)
- [Classifying COVID-19 vaccine narratives](https://mlsanity.com/paper/2207.08522/) (2022)
- [Reinforcement Learning for Topic Models](https://mlsanity.com/paper/2305.04843/) (2023)

## Abstract (arXiv)

Most real-world document collections involve various types of metadata, such as author, source, and date, and yet the most commonly-used approaches to modeling text corpora ignore this information. While specialized models have been developed for particular applications, few are widely used in practice, as customization typically requires derivation of a custom inference algorithm. In this paper, we build on recent advances in variational inference methods and propose a general neural framework, based on topic models, to enable flexible incorporation of metadata and allow for rapid exploration of alternative models. Our approach achieves strong performance, with a manageable tradeoff between perplexity, coherence, and sparsity. Finally, we demonstrate the potential of our framework through an exploration of a corpus of articles about US immigration.