# Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts

arXiv: https://arxiv.org/abs/2212.06002 · HTML: https://mlsanity.com/paper/2212.06002/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 12 Dec 2022.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence](https://mlsanity.com/paper/2004.03974/) (2020)
- [PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks](https://mlsanity.com/paper/1508.00200/) (2015)
- [Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence](https://mlsanity.com/paper/2107.02173/) (2021)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [Automated Phrase Mining from Massive Text Corpora](https://mlsanity.com/paper/1702.04457/) (2017)
- [Weakly-Supervised Neural Text Classification](https://mlsanity.com/paper/1809.01478/) (2018)
- [Topic Modeling with Contextualized Word Representation Clusters](https://mlsanity.com/paper/2010.12626/) (2020)
- [Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics](https://mlsanity.com/paper/2204.09874/) (2022)
- [Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations](https://mlsanity.com/paper/2202.04582/) (2022)
- [TaxoCom: Topic Taxonomy Completion with Hierarchical Discovery of Novel Topic Clusters](https://mlsanity.com/paper/2201.06771/) (2022)
- [Discriminative Topic Mining via Category-Name Guided Text Embedding](https://mlsanity.com/paper/1908.07162/) (2019)
- [Improving Topic Models with Latent Feature Word Representations](https://mlsanity.com/paper/1810.06306/) (2018)
- [Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge](https://mlsanity.com/paper/1611.10277/) (2016)
- [Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding](https://mlsanity.com/paper/2007.09536/) (2020)
- [HiGitClass: Keyword-Driven Hierarchical Classification of GitHub Repositories](https://mlsanity.com/paper/1910.07115/) (2019)
- [Nonparametric Spherical Topic Modeling with Word Embeddings](https://mlsanity.com/paper/1604.00126/) (2016)
- [MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information](https://mlsanity.com/paper/2111.04022/) (2021)
- [Keyword Assisted Embedded Topic Model](https://mlsanity.com/paper/2112.03101/) (2021)
- [Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds](https://mlsanity.com/paper/2205.01845/) (2022)

## Built on by (most cited first, 12 of 12)

- [Synthetic Data Generation Using Large Language Models: Advances in Text and Code](https://mlsanity.com/paper/2503.14023/) (2025)
- [Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations](https://mlsanity.com/paper/2408.15232/) (2024)
- [TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision](https://mlsanity.com/paper/2403.00165/) (2024)
- [Improving Retrieval in Theme-specific Applications using a Corpus Topical Taxonomy](https://mlsanity.com/paper/2403.04160/) (2024)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [MEGClass: Extremely Weakly Supervised Text Classification via Mutually-Enhancing Text Granularities](https://mlsanity.com/paper/2304.01969/) (2023)
- [Applying Text Mining to Analyze Human Question Asking in Creativity Research](https://mlsanity.com/paper/2501.02090/) (2025)
- [Topic Coverage-based Demonstration Retrieval for In-Context Learning](https://mlsanity.com/paper/2509.12451/) (2025)
- [MultiCube-RAG for Multi-hop Question Answering](https://mlsanity.com/paper/2602.15898/) (2026)
- [LLMSynthor: Macro-Aligned Micro-Records Synthesis with Large Language Models](https://mlsanity.com/paper/2505.14752/) (2025)
- [Seeded Poisson Factorization: leveraging domain knowledge to fit topic models](https://mlsanity.com/paper/2503.02741/) (2025)
- [Constrained Non-negative Matrix Factorization for Guided Topic Modeling of Minority Topics](https://mlsanity.com/paper/2505.16493/) (2025)

## Abstract (arXiv)

Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract distinctive and coherent topics so that the mined topics can better cater to the user's interest. To model the semantic correlation between words and seeds for discovering topic-indicative terms, existing seed-guided approaches utilize different types of context signals, such as document-level word co-occurrences, sliding window-based local contexts, and generic linguistic knowledge brought by pre-trained language models. In this work, we analyze and show empirically that each type of context information has its value and limitation in modeling word semantics under seed guidance, but combining three types of contexts (i.e., word embeddings learned from local contexts, pre-trained language model representations obtained from general-domain training, and topic-indicative sentences retrieved based on seed information) allows them to complement each other for discovering quality topics. We propose an iterative framework, SeedTopicMine, which jointly learns from the three types of contexts and gradually fuses their context signals via an ensemble ranking process. Under various sets of seeds and on multiple datasets, SeedTopicMine consistently yields more coherent and accurate topics than existing seed-guided topic discovery approaches.