# AutoTrial: Prompting Language Models for Clinical Trial Design

arXiv: https://arxiv.org/abs/2305.11366 · HTML: https://mlsanity.com/paper/2305.11366/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment](https://mlsanity.com/paper/2304.06767/) (2023)
- [RRHF: Rank Responses to Align Language Models with Human Feedback without tears](https://mlsanity.com/paper/2304.05302/) (2023)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [Deep Bidirectional Language-Knowledge Graph Pretraining](https://mlsanity.com/paper/2210.09338/) (2022)
- [Trial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision](https://mlsanity.com/paper/2206.14719/) (2022)

## Built on by (most cited first, 11 of 11)

- [Matching Patients to Clinical Trials with Large Language Models](https://mlsanity.com/paper/2307.15051/) (2023)
- [Panacea: A foundation model for clinical trial search, summarization, design, and recruitment](https://mlsanity.com/paper/2407.11007/) (2024)
- [From RAGs to riches: Utilizing large language models to write documents for clinical trials](https://mlsanity.com/paper/2402.16406/) (2024)
- [PyTrial: Machine Learning Software and Benchmark for Clinical Trial Applications](https://mlsanity.com/paper/2306.04018/) (2023)
- [CTBench: A Comprehensive Benchmark for Evaluating Language Model Capabilities in Clinical Trial Design](https://mlsanity.com/paper/2406.17888/) (2024)
- [InformGen: An AI Copilot for Accurate and Compliant Clinical Research Consent Document Generation](https://mlsanity.com/paper/2504.00934/) (2025)
- [Aligning Large Language Models with Healthcare Stakeholders: A Pathway to Trustworthy AI Integration](https://mlsanity.com/paper/2505.02848/) (2025)
- [Developing Large Language Models for Clinical Research Using One Million Clinical Trials](https://mlsanity.com/paper/2505.16097/) (2025)
- [Scalable Scientific Interest Profiling Using Large Language Models](https://mlsanity.com/paper/2508.15834/) (2025)
- [POET: Protocol Optimization via Eligibility Tuning](https://mlsanity.com/paper/2602.00370/) (2026)
- [TrialAtlas: Multi-Agent Research Organization for Clinical Trial Design and Optimization](https://mlsanity.com/paper/2609.21859/) (2026)

## Abstract (arXiv)

Clinical trials are critical for drug development. Constructing the appropriate eligibility criteria (i.e., the inclusion/exclusion criteria for patient recruitment) is essential for the trial's success. Proper design of clinical trial protocols should consider similar precedent trials and their eligibility criteria to ensure sufficient patient coverage. In this paper, we present a method named AutoTrial to aid the design of clinical eligibility criteria using language models. It allows (1) controllable generation under instructions via a hybrid of discrete and neural prompting, (2) scalable knowledge incorporation via in-context learning, and (3) explicit reasoning chains to provide rationales for understanding the outputs. Experiments on over 70K clinical trials verify that AutoTrial generates high-quality criteria texts that are fluent and coherent and with high accuracy in capturing the relevant clinical concepts to the target trial. It is noteworthy that our method, with a much smaller parameter size, gains around 60% winning rate against the GPT-3.5 baselines via human evaluations.