# Automatically Labeling Clinical Trial Outcomes: A Large-Scale Benchmark for Drug Development

arXiv: https://arxiv.org/abs/2406.10292 · HTML: https://mlsanity.com/paper/2406.10292/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 8 of 8)

- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [TimeLMs: Diachronic Language Models from Twitter](https://mlsanity.com/paper/2202.03829/) (2022)
- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Training Complex Models with Multi-Task Weak Supervision](https://mlsanity.com/paper/1810.02840/) (2018)
- [HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Data](https://mlsanity.com/paper/2102.04252/) (2021)
- [SPOT: Sequential Predictive Modeling of Clinical Trial Outcome with Meta-Learning](https://mlsanity.com/paper/2304.05352/) (2023)

## Built on by (most cited first, 7 of 7)

- [CT Open: An Open-Access, Uncontaminated, Live Platform for the Open Challenge of Clinical Trial Outcome Prediction](https://mlsanity.com/paper/2604.16742/) (2026)
- [MMCTOP: A Multimodal Textualization and Mixture-of-Experts Framework for Clinical Trial Outcome Prediction](https://mlsanity.com/paper/2512.21897/) (2025)
- [textttAMEND++: Benchmarking Eligibility Criteria Amendments in Clinical Trials](https://mlsanity.com/paper/2601.06300/) (2026)
- [DeepImagine: Clinical Trial Outcome Prediction via Stepwise Local Counterfactual Imaginations](https://mlsanity.com/paper/2604.23054/) (2026)
- [Position: Medical AI Neglects Real Treatment Outcomes](https://mlsanity.com/paper/2608.14598/) (2026)
- [TrialAtlas: Multi-Agent Research Organization for Clinical Trial Design and Optimization](https://mlsanity.com/paper/2609.21859/) (2026)
- [CHAIN: Calibrated LLM Forecasting via Causal-Temporal Hypergraph Inference](https://mlsanity.com/paper/2609.36689/) (2026)

## Abstract (arXiv)

Background The cost of drug discovery and development is substantial, with clinical trial outcomes playing a critical role in regulatory approval and patient care. However, access to large-scale, high-quality clinical trial outcome data remains limited, hindering advancements in predictive modeling and evidence-based decision-making. Methods We present the Clinical Trial Outcome (CTO) benchmark, a fully reproducible, large-scale repository encompassing approximately 125,000 drug and biologics trials. CTO integrates large language model (LLM) interpretations of publications, trial phase progression tracking, sentiment analysis from news sources, stock price movements of trial sponsors, and additional trial-related metrics. Furthermore, we manually annotated a dataset of clinical trials conducted between 2020 and 2024 to enhance the quality and reliability of outcome labels. Results The trial outcome labels in the CTO benchmark agree strongly with expert annotations, achieving an F1 score of 94 for Phase 3 trials and 91 across all phases. Additionally, benchmarking standard machine learning models on our manually annotated dataset revealed distribution shifts in recent trials, underscoring the necessity of continuously updated labeling approaches. Conclusions By analyzing CTO's performance on recent clinical trials, we demonstrate the ongoing need for high-quality, up-to-date trial outcome labels. We publicly release the CTO knowledge base and annotated labels at https://chufangao.github.io/CTOD, with regular updates to support research on clinical trial outcomes and inform data-driven improvements in drug development.