# Leveraging LLMs for Semi-Automatic Corpus Filtration in Systematic Literature Reviews

arXiv: https://arxiv.org/abs/2510.11409 · HTML: https://mlsanity.com/paper/2510.11409/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 53 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Builds on (most cited first, 16 of 16)

- [LitLLM: A Toolkit for Scientific Literature Review](https://mlsanity.com/paper/2402.01788/) (2024)
- [ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing](https://mlsanity.com/paper/2303.13367/) (2023)
- [AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews](https://mlsanity.com/paper/2408.10365/) (2024)
- [Benchmarking Prompt Sensitivity in Large Language Models](https://mlsanity.com/paper/2502.06065/) (2025)
- [Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning](https://mlsanity.com/paper/2404.08680/) (2024)
- [Artificial Intelligence for Literature Reviews: Opportunities and Challenges](https://mlsanity.com/paper/2402.08565/) (2024)
- [ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary](https://mlsanity.com/paper/2403.02574/) (2024)
- [The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic review](https://mlsanity.com/paper/2409.04600/) (2024)
- [System for systematic literature review using multiple AI agents: Concept and an empirical evaluation](https://mlsanity.com/paper/2403.08399/) (2024)
- [PRISMA-DFLLM: An Extension of PRISMA for Systematic Literature Reviews using Domain-specific Finetuned Large Language Models](https://mlsanity.com/paper/2306.14905/) (2023)
- [Cutting Through the Clutter: The Potential of LLMs for Efficient Filtration in Systematic Literature Reviews](https://mlsanity.com/paper/2407.10652/) (2024)
- [Promoting Ethical Awareness in Communication Analysis: Investigating Potentials and Limits of Visual Analytics for Intelligence Applications](https://mlsanity.com/paper/2203.09859/) (2022)
- [LLAssist: Simple Tools for Automating Literature Review Using Large Language Models](https://mlsanity.com/paper/2407.13993/) (2024)
- [Streamlining the Selection Phase of Systematic Literature Reviews (SLRs) Using AI-Enabled GPT-4 Assistant API](https://mlsanity.com/paper/2402.18582/) (2024)
- [Efficient Systematic Reviews: Literature Filtering with Transformers & Transfer Learning](https://mlsanity.com/paper/2405.20354/) (2024)
- [Visual Network Analysis in Immersive Environments: A Survey](https://mlsanity.com/paper/2501.08500/) (2025)

## Built on by (most cited first, 1 of 1)

- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)

## Abstract (arXiv)

The creation of systematic literature reviews (SLR) is critical for analyzing the landscape of a research field and guiding future research directions. However, retrieving and filtering the literature corpus for an SLR is highly time-consuming and requires extensive manual effort, as keyword-based searches in digital libraries often return numerous irrelevant publications. In this work, we propose a pipeline leveraging multiple large language models (LLMs), classifying papers based on descriptive prompts and deciding jointly using a consensus scheme. The entire process is human-supervised and interactively controlled via our open-source visual analytics web interface, LLMSurver, which enables real-time inspection and modification of model outputs. We evaluate our approach using ground-truth data from a recent SLR comprising over 8,000 candidate papers, benchmarking both open and commercial state-of-the-art LLMs from mid-2024 and fall 2025. Results demonstrate that our pipeline significantly reduces manual effort while achieving lower error rates than single human annotators. Furthermore, modern open-source models prove sufficient for this task, making the method accessible and cost-effective. Overall, our work demonstrates how responsible human-AI collaboration can accelerate and enhance systematic literature reviews within academic workflows.