# Accelerating Clinical Evidence Synthesis with Large Language Models

arXiv: https://arxiv.org/abs/2406.17755 · HTML: https://mlsanity.com/paper/2406.17755/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 85 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?](https://mlsanity.com/paper/2302.03495/) (2023)
- [Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarization](https://mlsanity.com/paper/2008.11293/) (2020)
- [Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)](https://mlsanity.com/paper/2305.06299/) (2023)
- [Appraising the Potential Uses and Harms of LLMs for Medical Systematic Reviews](https://mlsanity.com/paper/2305.11828/) (2023)
- [FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence](https://mlsanity.com/paper/2402.11456/) (2024)
- [PubMed and Beyond: Biomedical Literature Search in the Age of Artificial Intelligence](https://mlsanity.com/paper/2307.09683/) (2023)
- [Jointly Extracting Interventions, Outcomes, and Findings from RCT Reports with LLMs](https://mlsanity.com/paper/2305.03642/) (2023)
- [Assessing the Ability of ChatGPT to Screen Articles for Systematic Reviews](https://mlsanity.com/paper/2307.06464/) (2023)
- [Automatically Extracting Numerical Results from Randomized Controlled Trials with Large Language Models](https://mlsanity.com/paper/2405.01686/) (2024)
- [Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges](https://mlsanity.com/paper/2303.05392/) (2023)
- [PyTrial: Machine Learning Software and Benchmark for Clinical Trial Applications](https://mlsanity.com/paper/2306.04018/) (2023)

## Built on by (most cited first, 16 of 16)

- [A foundation model for human-AI collaboration in medical literature mining](https://mlsanity.com/paper/2501.16255/) (2025)
- [A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challenges](https://mlsanity.com/paper/2411.00024/) (2024)
- [Can Large Language Models Match the Conclusions of Systematic Reviews?](https://mlsanity.com/paper/2505.22787/) (2025)
- [Streamlining evidence based clinical recommendations with large language models](https://mlsanity.com/paper/2505.10282/) (2025)
- [Can Large Language Models Replace Data Scientists in Biomedical Research?](https://mlsanity.com/paper/2410.21591/) (2024)
- [DeepEvidence: Empowering Biomedical Discovery with Deep Knowledge Graph Research](https://mlsanity.com/paper/2601.11560/) (2025)
- [Developing Large Language Models for Clinical Research Using One Million Clinical Trials](https://mlsanity.com/paper/2505.16097/) (2025)
- [From Chaos to Clarity: Schema-Constrained AI for Auditable Biomedical Evidence Extraction from Full-Text PDFs](https://mlsanity.com/paper/2601.14267/) (2025)
- [The Agentification of Scientific Research: A Physicist's Perspective](https://mlsanity.com/paper/2604.14718/) (2026)
- [Biomedical reasoning in action: Multi-agent System for Auditable Biomedical Evidence Synthesis](https://mlsanity.com/paper/2510.05335/) (2025)
- [A PubMed-Scale Dataset of Structured Biomedical Abstracts](https://mlsanity.com/paper/2606.11361/) (2026)
- [LUMEN: Cost-Transparent Multi-Agent Pipeline for Automated Systematic Review and Meta-Analysis](https://mlsanity.com/paper/2606.28362/) (2026)
- [meta-pipe: An LLM-agent pipeline for end-to-end automated systematic review and meta-analysis](https://mlsanity.com/paper/2606.28363/) (2026)
- [TrialAtlas: Multi-Agent Research Organization for Clinical Trial Design and Optimization](https://mlsanity.com/paper/2609.21859/) (2026)
- [MedRouter: Demystifying Knowledge Differences Across Medical LLMs for Routing-Based Reasoning](https://mlsanity.com/paper/2609.33119/) (2026)
- [Answering clinicians' questions over trial evidence tables with verifiable, feedback-driven language models](https://mlsanity.com/paper/2610.02576/) (2026)

## Abstract (arXiv)

Synthesizing clinical evidence largely relies on systematic reviews of clinical trials and retrospective analyses from medical literature. However, the rapid expansion of publications presents challenges in efficiently identifying, summarizing, and updating clinical evidence. Here, we introduce TrialMind, a generative artificial intelligence (AI) pipeline for facilitating human-AI collaboration in three crucial tasks for evidence synthesis: study search, screening, and data extraction. To assess its performance, we chose published systematic reviews to build the benchmark dataset, named TrialReviewBench, which contains 100 systematic reviews and the associated 2,220 clinical studies. Our results show that TrialMind excels across all three tasks. In study search, it generates diverse and comprehensive search queries to achieve high recall rates (Ours 0.711-0.834 v.s. Human baseline 0.138-0.232). For study screening, TrialMind surpasses traditional embedding-based methods by 30% to 160%. In data extraction, it outperforms a GPT-4 baseline by 29.6% to 61.5%. We further conducted user studies to confirm its practical utility. Compared to manual efforts, human-AI collaboration using TrialMind yielded a 71.4% recall lift and 44.2% time savings in study screening and a 23.5% accuracy lift and 63.4% time savings in data extraction. Additionally, when comparing synthesized clinical evidence presented in forest plots, medical experts favored TrialMind's outputs over GPT-4's outputs in 62.5% to 100% of cases. These findings show the promise of LLM-based approaches like TrialMind to accelerate clinical evidence synthesis via streamlining study search, screening, and data extraction from medical literature, with exceptional performance improvement when working with human experts.