# LimTopic: LLM-based Topic Modeling and Text Summarization for Analyzing Scientific Articles limitations

arXiv: https://arxiv.org/abs/2503.10658 · HTML: https://mlsanity.com/paper/2503.10658/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 54 references that are in the map.
- First posted on arXiv 8 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [BooookScore: A systematic exploration of book-length summarization in the era of LLMs](https://mlsanity.com/paper/2310.00785/) (2023)
- [Prompting Large Language Models for Topic Modeling](https://mlsanity.com/paper/2312.09693/) (2023)
- [LimGen: Probing the LLMs for Generating Suggestive Limitations of Research Papers](https://mlsanity.com/paper/2403.15529/) (2024)
- [Measuring the Diversity of Facebook Reactions to Research](https://mlsanity.com/paper/2001.01029/) (2020)
- [Generate FAIR Literature Surveys with Scholarly Knowledge Graphs](https://mlsanity.com/paper/2006.01747/) (2020)
- [AHAM: Adapt, Help, Ask, Model -- Harvesting LLMs for literature mining](https://mlsanity.com/paper/2312.15784/) (2023)
- [L(u)PIN: LLM-based Political Ideology Nowcasting](https://mlsanity.com/paper/2405.07320/) (2024)

## Built on by (most cited first, 12 of 12)

- [Text summarization via global structure awareness](https://mlsanity.com/paper/2602.09821/) (2026)
- [LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Models](https://mlsanity.com/paper/2505.19240/) (2025)
- [Gen-Review: A Large-scale Dataset of AI-Generated (and Human-written) Peer Reviews](https://mlsanity.com/paper/2510.21192/) (2025)
- [FutureGen: A RAG-based Approach to Generate the Future Work of Scientific Article](https://mlsanity.com/paper/2503.16561/) (2025)
- [BAGELS: Benchmarking the Automated Generation and Extraction of Limitations from Scholarly Text](https://mlsanity.com/paper/2505.18207/) (2025)
- [Multi-Agent LLMs for Generating Research Limitations](https://mlsanity.com/paper/2601.11578/) (2025)
- [What Challenges Do Developers Face in AI Agent Systems? An Empirical Study on Stack Overflow & GitHub Issues](https://mlsanity.com/paper/2510.25423/) (2025)
- [Evolution of wartime discourse on Telegram: A comparative study of Ukrainian and Russian policymakers' communication before and after Russia's full-scale invasion of Ukraine](https://mlsanity.com/paper/2510.11746/) (2025)
- [ANCHOR: Abductive Network Construction with Hierarchical Orchestration for Reliable Probability Inference in Large Language Models](https://mlsanity.com/paper/2605.10328/) (2026)
- [DiSCo: Making Absence Visible in Intelligent Summarization Interfaces](https://mlsanity.com/paper/2601.07229/) (2026)
- [CheckSupport: A Local LLM-Powered Tool for Automated Manuscript Submission Checklist Selection and Completion](https://mlsanity.com/paper/2605.16377/) (2026)
- [What Limits Us? Analyzing Self-Reported Limitations in NLP Research](https://mlsanity.com/paper/2609.15191/) (2026)

## Abstract (arXiv)

The limitations sections of scientific articles play a crucial role in highlighting the boundaries and shortcomings of research, thereby guiding future studies and improving research methods. Analyzing these limitations benefits researchers, reviewers, funding agencies, and the broader academic community. We introduce LimTopic, a strategy where Topic generation in Limitation sections in scientific articles with Large Language Models (LLMs). Here, each topic contains the title and Topic Summary. This study focuses on effectively extracting and understanding these limitations through topic modeling and text summarization, utilizing the capabilities of LLMs. We extracted limitations from research articles and applied an LLM-based topic modeling integrated with the BERtopic approach to generate a title for each topic and Topic Sentences. To enhance comprehension and accessibility, we employed LLM-based text summarization to create concise and generalizable summaries for each topic Topic Sentences and produce a Topic Summary. Our experimentation involved prompt engineering, fine-tuning LLM and BERTopic, and integrating BERTopic with LLM to generate topics, titles, and a topic summary. We also experimented with various LLMs with BERTopic for topic modeling and various LLMs for text summarization tasks. Our results showed that the combination of BERTopic and GPT 4 performed the best in terms of silhouette and coherence scores in topic modeling, and the GPT4 summary outperformed other LLM tasks as a text summarizer.