# SciTopic: Enhancing Topic Discovery in Scientific Literature through Advanced LLM

arXiv: https://arxiv.org/abs/2508.20514 · HTML: https://mlsanity.com/paper/2508.20514/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Autoencoding Variational Inference For Topic Models](https://mlsanity.com/paper/1703.01488/) (2017)
- [Neural Variational Inference for Text Processing](https://mlsanity.com/paper/1511.06038/) (2015)
- [Iterative Prompt Learning for Unsupervised Backlit Image Enhancement](https://mlsanity.com/paper/2303.17569/) (2023)
- [Graph Prompt Learning: A Comprehensive Survey and Beyond](https://mlsanity.com/paper/2311.16534/) (2023)
- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Neural Topic Model via Optimal Transport](https://mlsanity.com/paper/2008.13537/) (2020)
- [Effective Neural Topic Modeling with Embedding Clustering Regularization](https://mlsanity.com/paper/2306.04217/) (2023)
- [When Prompt-based Incremental Learning Does Not Meet Strong Pretraining](https://mlsanity.com/paper/2308.10445/) (2023)
- [You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content](https://mlsanity.com/paper/2308.05596/) (2023)
- [Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning](https://mlsanity.com/paper/2211.12878/) (2022)
- [Cross-Batch Negative Sampling for Training Two-Tower Recommenders](https://mlsanity.com/paper/2110.15154/) (2021)
- [InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling](https://mlsanity.com/paper/2304.03544/) (2023)
- [Domain-Controlled Prompt Learning](https://mlsanity.com/paper/2310.07730/) (2023)
- [Match-Prompt: Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning](https://mlsanity.com/paper/2204.02725/) (2022)
- [Neural Topic Modeling with Large Language Models in the Loop](https://mlsanity.com/paper/2411.08534/) (2024)
- [scSiameseClu: A Siamese Clustering Framework for Interpreting single-cell RNA Sequencing Data](https://mlsanity.com/paper/2505.12626/) (2025)
- [Rethinking Graph Contrastive Learning through Relative Similarity Preservation](https://mlsanity.com/paper/2505.05533/) (2025)
- [Scientific Dataset Discovery via Topic-level Recommendation](https://mlsanity.com/paper/2106.03399/) (2021)

## Built on by (most cited first, 1 of 1)

- [Mapping Scientific Literature with Large Language Models and Topic Modeling](https://mlsanity.com/paper/2510.16152/) (2025)

## Abstract (arXiv)

Topic discovery in scientific literature provides valuable insights for researchers to identify emerging trends and explore new avenues for investigation, facilitating easier scientific information retrieval. Many machine learning methods, particularly deep embedding techniques, have been applied to discover research topics. However, most existing topic discovery methods rely on word embedding to capture the semantics and lack a comprehensive understanding of scientific publications, struggling with complex, high-dimensional text relationships. Inspired by the exceptional comprehension of textual information by large language models (LLMs), we propose an advanced topic discovery method enhanced by LLMs to improve scientific topic identification, namely SciTopic. Specifically, we first build a textual encoder to capture the content from scientific publications, including metadata, title, and abstract. Next, we construct a space optimization module that integrates entropy-based sampling and triplet tasks guided by LLMs, enhancing the focus on thematic relevance and contextual intricacies between ambiguous instances. Then, we propose to fine-tune the textual encoder based on the guidance from the LLMs by optimizing the contrastive loss of the triplets, forcing the text encoder to better discriminate instances of different topics. Finally, extensive experiments conducted on three real-world datasets of scientific publications demonstrate that SciTopic outperforms the state-of-the-art (SOTA) scientific topic discovery methods, enabling researchers to gain deeper and faster insights.