mlsanityData to 7 Oct 2026

Automated Paper Screening for Clinical Reviews Using Large Language Models

arXiv 2305.00844 ↗First posted 1 May 2023cs.CL cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

Objective: To assess the performance of the OpenAI GPT API in accurately and efficiently identifying relevant titles and abstracts from real-world clinical review datasets and compare its performance against ground truth labelling by two independent human reviewers. Methods: We introduce a novel workflow using the OpenAI GPT API for screening titles and abstracts in clinical reviews. A Python script was created to make calls to the GPT API with the screening criteria in natural language and a corpus of title and abstract datasets that have been filtered by a minimum of two human reviewers. We compared the performance of our model against human-reviewed papers across six review papers, screening over 24,000 titles and abstracts. Results: Our results show an accuracy of 0.91, a sensitivity of excluded papers of 0.91, and a sensitivity of included papers of 0.76. On a randomly selected subset of papers, the GPT API demonstrated the ability to provide reasoning for its decisions and corrected its initial decision upon being asked to explain its reasoning for a subset of incorrect classifications. Conclusion: The GPT API has the potential to streamline the clinical review process, save valuable time and effort for researchers, and contribute to the overall quality of clinical reviews. By prioritizing the workflow and acting as an aid rather than a replacement for researchers and reviewers, the GPT API can enhance efficiency and lead to more accurate and reliable conclusions in medical research.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 2 of its 42 references that are in the map. First posted on arXiv 1 May 2023.

Builds on

2 of 42 references.

References · 2

  1. 2020Open Source Software for Efficient and Transparent Reviewscited by 34
  2. 2020Trialstreamer: Mapping and Browsing Medical Evidence in Real-Timecited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 9; 2026: 8.

29 papers.

Sort:

By area:

  1. 2025Large language models for automated scholarly paper review: A surveycited by 59
  2. 2023Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languagescited by 36
  3. 2024The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic reviewcited by 20
  4. 2024A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industrycited by 17
  5. 2024The Promise and Challenges of Using LLMs to Accelerate the Screening Process of Systematic Reviewscited by 14
  6. 2024Toward Automated Simulation Research Workflow through LLM Prompt Engineering Designcited by 10
  7. 2023Bio-SIEVE: Exploring Instruction Tuning Large Language Models for Systematic Review Automationcited by 6
  8. 2024RelevAI-Reviewer: A Benchmark on AI Reviewers for Survey Paper Relevancecited by 6
  9. 2023Automated title and abstract screening for scoping reviews using the GPT-4 Large Language Modelcited by 5
  10. 2025LLM4SCREENLIT: Recommendations on Assessing the Performance of Large Language Models for Screening Literature in Systematic Reviewscited by 5
  11. 2024Measuring Risk of Bias in Biomedical Reports: The RoBBR Benchmarkcited by 4
  12. 2024Streamlining Systematic Reviews: A Novel Application of Large Language Modelscited by 4
17 more
  1. 2025Large Language Models for Healthcare Text Classification: A Systematic Reviewcited by 3
  2. 2025Evaluating Prompting Strategies and Large Language Models in Systematic Literature Review Screening: Relevance and Task-Stage Classificationcited by 3
  3. 2026AutoSynthesis: An agentic system for automated meta-analysiscited by 2
  4. 2024Generative AI in Medicinecited by 1
  5. 2025Large language models streamline automated systematic review: A preliminary studycited by 1
  6. 2026Jev in Medicine: A Benchmark Evaluationcited by 1
  7. 2024Assisting humans in complex comparisons: automated information comparison at scalecited by 0
  8. 2025Enhancing LLMs for Identifying and Prioritizing Important Medical Jargons from Electronic Health Record Notes Utilizing Data Augmentation: A Comparative Studycited by 0
  9. 2025CKD-EHR:Clinical Knowledge Distillation for Electronic Health Recordscited by 0
  10. 2025An Auditable Pipeline for Fuzzy Full-Text Screening in Systematic Reviews: Integrating Contrastive Semantic Highlighting and LLM Judgmentcited by 0
  11. 2025Leveraging LLMs for Title and Abstract Screening for Systematic Review: A Cost-Effective Dynamic Few-Shot Learning Approachcited by 0
  12. 2026Improving Behavioral Alignment in LLM Social Simulations via Context Formation and Navigationcited by 0
  13. 2026TiAb Review Plugin: A Browser-Based Tool for AI-Assisted Study Selection in Systematic Reviewscited by 0
  14. 2026A Tertiary Review of Large Language Model-Based Code Generating Tasks: Trends, Challenges, and Future Directionscited by 0
  15. 2026Evaluation design conditions the expert-vs-auto MeSH gap: a controlled comparison of bag-of-words and BiomedBERT on the Cohen benchmarkcited by 0
  16. 2026Auxiliary uncertainty signals for LLM-assisted systematic review screening: a benchmark across eight Cohen drug-class reviewscited by 0
  17. 2026Using Human-LLM Disagreement to Improve Checklist-Based Quality Appraisalcited by 0