# Topically Driven Neural Language Model

arXiv: https://arxiv.org/abs/1704.08012 · HTML: https://mlsanity.com/paper/1704.08012/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2017.

## Built on by (most cited first, 8 of 8)

- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Learning to Plan for Language Modeling from Unlabeled Data](https://mlsanity.com/paper/2404.00614/) (2024)
- [Are Neural Topic Models Broken?](https://mlsanity.com/paper/2210.16162/) (2022)
- [Multi-turn Dialogue Comprehension from a Topic-aware Perspective](https://mlsanity.com/paper/2309.09666/) (2023)
- [Revisiting Topic-Guided Language Models](https://mlsanity.com/paper/2312.02331/) (2023)
- [TopNet: Learning from Neural Topic Model to Generate Long Stories](https://mlsanity.com/paper/2112.07259/) (2021)
- [Topical Language Generation using Transformers](https://mlsanity.com/paper/2103.06434/) (2021)
- [Recurrent Coupled Topic Modeling over Sequential Documents](https://mlsanity.com/paper/2106.13732/) (2021)

## Abstract (arXiv)

Language models are typically applied at the sentence level, without access to the broader document context. We present a neural language model that incorporates document context in the form of a topic model-like architecture, thus providing a succinct representation of the broader document context outside of the current sentence. Experiments over a range of datasets demonstrate that our model outperforms a pure sentence-based model in terms of language model perplexity, and leads to topics that are potentially more coherent than those produced by a standard LDA topic model. Our model also has the ability to generate related sentences for a topic, providing another way to interpret topics.