# LeafAI: query generator for clinical cohort discovery rivaling a human programmer

arXiv: https://arxiv.org/abs/2304.06203 · HTML: https://mlsanity.com/paper/2304.06203/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 78 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Text-to-SQL Generation for Question Answering on Electronic Medical Records](https://mlsanity.com/paper/1908.01839/) (2019)
- [Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2001.11458/) (2020)
- [Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations](https://mlsanity.com/paper/2104.07478/) (2021)
- [Knowledge Graph-based Question Answering with Electronic Health Records](https://mlsanity.com/paper/2010.09394/) (2020)
- [Improving Semantic Parsing for Task Oriented Dialog](https://mlsanity.com/paper/1902.06000/) (2019)
- [The Leaf Clinical Trials Corpus: a new resource for query generation from clinical trial eligibility criteria](https://mlsanity.com/paper/2207.13757/) (2022)
- [Question Answering for Complex Electronic Health Records Database using Unified Encoder-Decoder Architecture](https://mlsanity.com/paper/2111.14703/) (2021)
- [Patient Cohort Retrieval using Transformer Language Models](https://mlsanity.com/paper/2009.05121/) (2020)

## Built on by (most cited first, 9 of 9)

- [EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images](https://mlsanity.com/paper/2310.18652/) (2023)
- [PhenoFlow: A Human-LLM Driven Visual Analytics System for Exploring Large and Complex Stroke Datasets](https://mlsanity.com/paper/2407.16329/) (2024)
- [Generalizable and Scalable Multistage Biomedical Concept Normalization Leveraging Large Language Models](https://mlsanity.com/paper/2405.15122/) (2024)
- [Automated Survey Collection with LLM-based Conversational Agents](https://mlsanity.com/paper/2504.02891/) (2025)
- [Generating patient cohorts from electronic health records using two-step retrieval-augmented text-to-SQL generation](https://mlsanity.com/paper/2502.21107/) (2025)
- [CACER: Clinical Concept Annotations for Cancer Events and Relations](https://mlsanity.com/paper/2409.03905/) (2024)
- [Natural Language Processing in Support of Evidence-based Medicine: A Scoping Review](https://mlsanity.com/paper/2505.22280/) (2025)
- [Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Disease](https://mlsanity.com/paper/2505.23852/) (2025)
- [Ontology-Based Post-Coordinated Expression Assembly: from Clinical Narrative to Queryable Knowledge Graphs](https://mlsanity.com/paper/2610.16474/) (2026)

## Abstract (arXiv)

Objective: Identifying study-eligible patients within clinical databases is a critical step in clinical research. However, accurate query design typically requires extensive technical and biomedical expertise. We sought to create a system capable of generating data model-agnostic queries while also providing novel logical reasoning capabilities for complex clinical trial eligibility criteria. Materials and Methods: The task of query creation from eligibility criteria requires solving several text-processing problems, including named entity recognition and relation extraction, sequence-to-sequence transformation, normalization, and reasoning. We incorporated hybrid deep learning and rule-based modules for these, as well as a knowledge base of the Unified Medical Language System (UMLS) and linked ontologies. To enable data-model agnostic query creation, we introduce a novel method for tagging database schema elements using UMLS concepts. To evaluate our system, called LeafAI, we compared the capability of LeafAI to a human database programmer to identify patients who had been enrolled in 8 clinical trials conducted at our institution. We measured performance by the number of actual enrolled patients matched by generated queries. Results: LeafAI matched a mean 43% of enrolled patients with 27,225 eligible across 8 clinical trials, compared to 27% matched and 14,587 eligible in queries by a human database programmer. The human programmer spent 26 total hours crafting queries compared to several minutes by LeafAI. Conclusions: Our work contributes a state-of-the-art data model-agnostic query generation system capable of conditional reasoning using a knowledge base. We demonstrate that LeafAI can rival an experienced human programmer in finding patients eligible for clinical trials.