# UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining

arXiv: https://arxiv.org/abs/2202.13469 · HTML: https://mlsanity.com/paper/2202.13469/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 27 Feb 2022.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [CLEAR: Contrastive Learning for Sentence Representation](https://mlsanity.com/paper/2012.15466/) (2020)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining](https://mlsanity.com/paper/2102.08473/) (2021)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [Learning Dense Representations of Phrases at Scale](https://mlsanity.com/paper/2012.12624/) (2020)
- [Supporting Clustering with Contrastive Learning](https://mlsanity.com/paper/2103.12953/) (2021)
- [On Sampling Strategies for Neural Network-based Collaborative Filtering](https://mlsanity.com/paper/1706.07881/) (2017)
- [Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration](https://mlsanity.com/paper/2109.06304/) (2021)
- [KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents](https://mlsanity.com/paper/1911.12559/) (2019)
- [Self-Taught Convolutional Neural Networks for Short Text Clustering](https://mlsanity.com/paper/1701.00185/) (2017)
- [Empower Entity Set Expansion via Language Model Probing](https://mlsanity.com/paper/2004.13897/) (2020)
- [Embarrassingly Simple Unsupervised Aspect Extraction](https://mlsanity.com/paper/2004.13580/) (2020)
- [Abstractive Multi-Document Summarization via Phrase Selection and Merging](https://mlsanity.com/paper/1506.01597/) (2015)
- [Scalable Topical Phrase Mining from Text Corpora](https://mlsanity.com/paper/1406.6312/) (2014)
- [Learning Phrase Embeddings from Paraphrases with GRUs](https://mlsanity.com/paper/1710.05094/) (2017)

## Built on by (most cited first, 30 of 33)

- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [XRec: Large Language Models for Explainable Recommendation](https://mlsanity.com/paper/2406.02377/) (2024)
- [G-Refer: Graph Retrieval-Augmented Large Language Model for Explainable Recommendation](https://mlsanity.com/paper/2502.12586/) (2025)
- [EditLens: Quantifying the Extent of AI Editing in Text](https://mlsanity.com/paper/2510.03154/) (2025)
- [TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics](https://mlsanity.com/paper/2502.02975/) (2025)
- [Searching for PETs: Using Distributional and Sentiment-Based Methods to Find Potentially Euphemistic Terms](https://mlsanity.com/paper/2205.10451/) (2022)
- [Topic Modeling as Multi-Objective Contrastive Optimization](https://mlsanity.com/paper/2402.07577/) (2024)
- [UCEpic: Unifying Aspect Planning and Lexical Constraints for Generating Explanations in Recommendation](https://mlsanity.com/paper/2209.13885/) (2022)
- [General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generation](https://mlsanity.com/paper/2208.09606/) (2022)
- [Beyond Static Testbeds: An Interaction-Centric Agent Simulation Platform for Dynamic Recommender Systems](https://mlsanity.com/paper/2505.16429/) (2025)
- [Learning High-Quality and General-Purpose Phrase Representations](https://mlsanity.com/paper/2401.10407/) (2024)
- [CLIP also Understands Text: Prompting CLIP for Phrase Understanding](https://mlsanity.com/paper/2210.05836/) (2022)
- [Fragment and Integrate Network (FIN): A Novel Spatial-Temporal Modeling Based on Long Sequential Behavior for Online Food Ordering Click-Through Rate Prediction](https://mlsanity.com/paper/2308.15703/) (2023)
- [Effective Edge-wise Representation Learning in Edge-Attributed Bipartite Graphs](https://mlsanity.com/paper/2406.13369/) (2024)
- ["Newspaper Eat" Means "Not Tasty": A Taxonomy and Benchmark for Coded Language in Real-World Chinese Online Reviews](https://mlsanity.com/paper/2601.19932/) (2026)
- [EnCore: Fine-Grained Entity Typing by Pre-Training Entity Encoders on Coreference Chains](https://mlsanity.com/paper/2305.12924/) (2023)
- [Phrase Retrieval for Open-Domain Conversational Question Answering with Conversational Dependency Modeling via Contrastive Learning](https://mlsanity.com/paper/2306.04293/) (2023)
- [Enhancing Neural Machine Translation with Semantic Units](https://mlsanity.com/paper/2310.11360/) (2023)
- [Retrieval-Augmented Recommendation Explanation Generation with Hierarchical Aggregation](https://mlsanity.com/paper/2507.09188/) (2025)
- [Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?](https://mlsanity.com/paper/2510.16060/) (2025)
- [Open-world Semi-supervised Generalized Relation Discovery Aligned in a Real-world Setting](https://mlsanity.com/paper/2305.13533/) (2023)
- [Graph Contrastive Topic Model](https://mlsanity.com/paper/2307.02078/) (2023)
- [Contrastive Learning-based Imputation-Prediction Networks for In-hospital Mortality Risk Modeling using EHRs](https://mlsanity.com/paper/2308.09896/) (2023)
- [Object Recognition from Scientific Document based on Compartment Refinement Framework](https://mlsanity.com/paper/2312.09038/) (2023)
- [KEPLET: Knowledge-Enhanced Pretrained Language Model with Topic Entity Awareness](https://mlsanity.com/paper/2305.01810/) (2023)
- [VisualLens: Personalization through Task-Agnostic Visual History](https://mlsanity.com/paper/2411.16034/) (2024)
- [Toward Equitable Access: Leveraging Crowdsourced Reviews to Investigate Public Perceptions of Health Resource Accessibility](https://mlsanity.com/paper/2502.10641/) (2025)
- [Reproducibility Study of "XRec: Large Language Models for Explainable Recommendation"](https://mlsanity.com/paper/2510.06275/) (2025)
- [ViCLSR: A Supervised Contrastive Learning Framework with Natural Language Inference for Natural Language Understanding Tasks](https://mlsanity.com/paper/2603.21084/) (2026)
- [Bridging Textual Profiles and Latent User Embeddings for Personalization](https://mlsanity.com/paper/2605.06981/) (2026)

## Abstract (arXiv)

High-quality phrase representations are essential to finding topics and related terms in documents (a.k.a. topic mining). Existing phrase representation learning methods either simply combine unigram representations in a context-free manner or rely on extensive annotations to learn context-aware knowledge. In this paper, we propose UCTopic, a novel unsupervised contrastive learning framework for context-aware phrase representations and topic mining. UCTopic is pretrained in a large scale to distinguish if the contexts of two phrase mentions have the same semantics. The key to pretraining is positive pair construction from our phrase-oriented assumptions. However, we find traditional in-batch negatives cause performance decay when finetuning on a dataset with small topic numbers. Hence, we propose cluster-assisted contrastive learning(CCL) which largely reduces noisy negatives by selecting negatives from clusters and further improves phrase representations for topics accordingly. UCTopic outperforms the state-of-the-art phrase representation model by 38.2% NMI in average on four entity cluster-ing tasks. Comprehensive evaluation on topic mining shows that UCTopic can extract coherent and diverse topical phrases.