# Zero-Shot Clinical Trial Patient Matching with LLMs

arXiv: https://arxiv.org/abs/2402.05125 · HTML: https://mlsanity.com/paper/2402.05125/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 56 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [YaRN: Efficient Context Window Extension of Large Language Models](https://mlsanity.com/paper/2309.00071/) (2023)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Med-HALT: Medical Domain Hallucination Test for Large Language Models](https://mlsanity.com/paper/2307.15343/) (2023)
- [Towards Conversational Diagnostic AI](https://mlsanity.com/paper/2401.05654/) (2024)
- [LLMs Accelerate Annotation for Medical Information Extraction](https://mlsanity.com/paper/2312.02296/) (2023)
- [Matching Patients to Clinical Trials with Large Language Models](https://mlsanity.com/paper/2307.15051/) (2023)
- [MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records](https://mlsanity.com/paper/2308.14089/) (2023)
- [COMPOSE: Cross-Modal Pseudo-Siamese Network for Patient Trial Matching](https://mlsanity.com/paper/2006.08765/) (2020)
- [Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology](https://mlsanity.com/paper/2308.02180/) (2023)
- [Distilling Large Language Models for Matching Patients to Clinical Trials](https://mlsanity.com/paper/2312.09958/) (2023)
- [TRIALSCOPE: A Unifying Causal Framework for Scaling Real-World Evidence Generation with Biomedical Language Models](https://mlsanity.com/paper/2311.01301/) (2023)
- [Comparing Rule-Based and Deep Learning Models for Patient Phenotyping](https://mlsanity.com/paper/1703.08705/) (2017)

## Built on by (most cited first, 30 of 35)

- [Advancing Multimodal Medical Capabilities of Gemini](https://mlsanity.com/paper/2405.03162/) (2024)
- [MedCalc-Bench: Evaluating Large Language Models for Medical Calculations](https://mlsanity.com/paper/2406.12036/) (2024)
- [Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions](https://mlsanity.com/paper/2408.00727/) (2024)
- [BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments](https://mlsanity.com/paper/2405.17631/) (2024)
- [BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature](https://mlsanity.com/paper/2501.07171/) (2025)
- [ClinicalMamba: A Generative Clinical Language Model on Longitudinal Clinical Notes](https://mlsanity.com/paper/2403.05795/) (2024)
- [iText2KG: Incremental Knowledge Graphs Construction Using Large Language Models](https://mlsanity.com/paper/2409.03284/) (2024)
- [A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challenges](https://mlsanity.com/paper/2411.00024/) (2024)
- [A Collection of Innovations in Medical AI for patient records in 2024](https://mlsanity.com/paper/2503.05768/) (2025)
- [MultiCast: Zero-Shot Multivariate Time Series Forecasting Using LLMs](https://mlsanity.com/paper/2405.14748/) (2024)
- [Automated Clinical Data Extraction with Knowledge Conditioned LLMs](https://mlsanity.com/paper/2406.18027/) (2024)
- [PRISM: Patient Records Interpretation for Semantic Clinical Trial Matching using Large Language Models](https://mlsanity.com/paper/2404.15549/) (2024)
- [Medical Triage as Pairwise Ranking: A Benchmark for Urgency in Patient Portal Messages](https://mlsanity.com/paper/2601.13178/) (2026)
- [Entry-level guide to the use of large language models for medical research](https://mlsanity.com/paper/2410.18856/) (2024)
- [Clinical trial cohort selection using Large Language Models on n2c2 Challenges](https://mlsanity.com/paper/2501.11114/) (2025)
- [LLM-Match: An Open-Sourced Patient Matching Model Based on Large Language Models and Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.13281/) (2025)
- [Real-world validation of a multimodal LLM-powered pipeline for High-Accuracy Clinical Trial Patient Matching leveraging EHR data](https://mlsanity.com/paper/2503.15374/) (2025)
- [A Survey on LLM-Assisted Clinical Trial Recruitment](https://mlsanity.com/paper/2506.15301/) (2025)
- [Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution](https://mlsanity.com/paper/2603.05308/) (2026)
- [Generative AI in Medicine](https://mlsanity.com/paper/2412.10337/) (2024)
- [Embedding-Driven Diversity Sampling to Improve Few-Shot Synthetic Data Generation](https://mlsanity.com/paper/2501.11199/) (2025)
- [Comparing representations of long clinical texts for the task of patient note-identification](https://mlsanity.com/paper/2503.24006/) (2025)
- [Developing Large Language Models for Clinical Research Using One Million Clinical Trials](https://mlsanity.com/paper/2505.16097/) (2025)
- [Are Multimodal LLMs Ready for Clinical Dermatology? A Real-World Evaluation in Dermatology](https://mlsanity.com/paper/2605.04098/) (2026)
- [Am I eligible? Natural Language Inference for Clinical Trial Patient Recruitment: the Patient's Point of View](https://mlsanity.com/paper/2503.15718/) (2025)
- [Initial Investigation of LLM-Assisted Development of Rule-Based Clinical NLP System](https://mlsanity.com/paper/2506.16628/) (2025)
- [A systematic review of trial-matching pipelines using large language models](https://mlsanity.com/paper/2509.19327/) (2025)
- [AI-assisted workflow enables rapid, high-fidelity breast cancer clinical trial eligibility prescreening](https://mlsanity.com/paper/2511.05696/) (2025)
- [DiscoVerse: Multi-Agent Pharmaceutical Co-Scientist for Traceable Drug Discovery and Reverse Translation](https://mlsanity.com/paper/2511.18259/) (2025)
- [Eligibility-Aware Evidence Synthesis: An Agentic Framework for Clinical Trial Meta-Analysis](https://mlsanity.com/paper/2604.02678/) (2026)

## Abstract (arXiv)

Matching patients to clinical trials is a key unsolved challenge in bringing new drugs to market. Today, identifying patients who meet a trial's eligibility criteria is highly manual, taking up to 1 hour per patient. Automated screening is challenging, however, as it requires understanding unstructured clinical text. Large language models (LLMs) offer a promising solution. In this work, we explore their application to trial matching. First, we design an LLM-based system which, given a patient's medical history as unstructured clinical text, evaluates whether that patient meets a set of inclusion criteria (also specified as free text). Our zero-shot system achieves state-of-the-art scores on the n2c2 2018 cohort selection benchmark. Second, we improve the data and cost efficiency of our method by identifying a prompting strategy which matches patients an order of magnitude faster and more cheaply than the status quo, and develop a two-stage retrieval pipeline that reduces the number of tokens processed by up to a third while retaining high performance. Third, we evaluate the interpretability of our system by having clinicians evaluate the natural language justifications generated by the LLM for each eligibility decision, and show that it can output coherent explanations for 97% of its correct decisions and 75% of its incorrect ones. Our results establish the feasibility of using LLMs to accelerate clinical trial operations.