# TRIALSCOPE: A Unifying Causal Framework for Scaling Real-World Evidence Generation with Biomedical Language Models

arXiv: https://arxiv.org/abs/2311.01301 · HTML: https://mlsanity.com/paper/2311.01301/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 58 references that are in the map.
- First posted on arXiv 2 Nov 2023.

## Builds on (most cited first, 10 of 10)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [DoWhy: An End-to-End Library for Causal Inference](https://mlsanity.com/paper/2011.04216/) (2020)
- [Matching Patients to Clinical Trials with Large Language Models](https://mlsanity.com/paper/2307.15051/) (2023)
- [Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology](https://mlsanity.com/paper/2308.02180/) (2023)
- [Characterization of Overlap in Observational Studies](https://mlsanity.com/paper/1907.04138/) (2019)
- [Multi-disciplinary fairness considerations in machine learning for clinical trials](https://mlsanity.com/paper/2205.08875/) (2022)
- [Towards Structuring Real-World Data at Scale: Deep Learning for Extracting Key Oncology Information from Clinical Text with Patient-Level Supervision](https://mlsanity.com/paper/2203.10442/) (2022)
- [Randomized trials and their observational emulations: a framework for benchmarking and joint analysis](https://mlsanity.com/paper/2203.14857/) (2022)
- [A sensitivity analysis approach for the causal hazard ratio in randomized and observational studies](https://mlsanity.com/paper/2202.12420/) (2022)

## Built on by (most cited first, 7 of 7)

- [Zero-Shot Clinical Trial Patient Matching with LLMs](https://mlsanity.com/paper/2402.05125/) (2024)
- [Reasoning Elicitation in Language Models via Counterfactual Feedback](https://mlsanity.com/paper/2410.03767/) (2024)
- [Efficient Randomized Experiments Using Foundation Models](https://mlsanity.com/paper/2502.04262/) (2025)
- [Universal Abstraction: Harnessing Frontier Models to Structure Real-World Data at Scale](https://mlsanity.com/paper/2502.00943/) (2025)
- [LLM-Match: An Open-Sourced Patient Matching Model Based on Large Language Models and Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.13281/) (2025)
- [Beyond Correlation: Towards Causal Large Language Model Agents in Biomedicine](https://mlsanity.com/paper/2505.16982/) (2025)
- [TrialCalibre: A Fully Automated Causal Engine for RCT Benchmarking and Observational Trial Calibration](https://mlsanity.com/paper/2604.25832/) (2026)

## Abstract (arXiv)

The rapid digitization of real-world data presents an unprecedented opportunity to optimize healthcare delivery and accelerate biomedical discovery. However, these data are often found in unstructured forms such as clinical notes in electronic medical records (EMRs), and is typically plagued by confounders, making it challenging to generate robust real-world evidence (RWE). Therefore, we present TRIALSCOPE, a framework designed to distil RWE from population level observational data at scale. TRIALSCOPE leverages biomedical language models to structure clinical text at scale, employs advanced probabilistic modeling for denoising and imputation, and incorporates state-of-the-art causal inference techniques to address common confounders in treatment effect estimation. Extensive experiments were conducted on a large-scale dataset of over one million cancer patients from a single large healthcare network in the United States. TRIALSCOPE was shown to automatically curate high-quality structured patient data, expanding the dataset and incorporating key patient attributes only available in unstructured form. The framework reduces confounding in treatment effect estimation, generating comparable results to randomized controlled lung cancer trials. Additionally, we demonstrate simulations of unconducted clinical trials - including a pancreatic cancer trial with varying eligibility criteria - using a suite of validation tests to ensure robustness. Thorough ablation studies were conducted to better understand key components of TRIALSCOPE and establish best practices for RWE generation from EMRs. TRIALSCOPE was able to extract data cancer treatment data from EMRs, overcoming limitations of manual curation. We were also able to show that TRIALSCOPE could reproduce results of lung and pancreatic cancer clinical trials from the extracted real world data.