# Inferring Which Medical Treatments Work from Reports of Clinical Trials

arXiv: https://arxiv.org/abs/1904.01606 · HTML: https://mlsanity.com/paper/1904.01606/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2019.

## Built on by (most cited first, 30 of 64)

- [Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2](https://mlsanity.com/paper/2311.10702/) (2023)
- [MultiVerS: Improving scientific claim verification with weak supervision and full-document context](https://mlsanity.com/paper/2112.01640/) (2021)
- [Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing](https://mlsanity.com/paper/2102.12060/) (2021)
- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [MS2: Multi-Document Summarization of Medical Studies](https://mlsanity.com/paper/2104.06486/) (2021)
- [Generating Scientific Claims for Zero-Shot Scientific Fact Checking](https://mlsanity.com/paper/2203.12990/) (2022)
- [Your fairness may vary: Pretrained language model fairness in toxic text classification](https://mlsanity.com/paper/2108.01250/) (2021)
- [Understanding Interlocking Dynamics of Cooperative Rationalization](https://mlsanity.com/paper/2110.13880/) (2021)
- [On the Diversity and Limits of Human Explanations](https://mlsanity.com/paper/2106.11988/) (2021)
- [Explain and Predict, and then Predict Again](https://mlsanity.com/paper/2101.04109/) (2021)
- [What to Learn, and How: Toward Effective Learning from Rationales](https://mlsanity.com/paper/2112.00071/) (2021)
- [CREST: A Joint Framework for Rationalization and Counterfactual Text Generation](https://mlsanity.com/paper/2305.17075/) (2023)
- [Literature-Augmented Clinical Outcome Prediction](https://mlsanity.com/paper/2111.08374/) (2021)
- [MDACE: MIMIC Documents Annotated with Code Evidence](https://mlsanity.com/paper/2307.03859/) (2023)
- [Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluations](https://mlsanity.com/paper/2305.13693/) (2023)
- [FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence](https://mlsanity.com/paper/2402.11456/) (2024)
- [Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience](https://mlsanity.com/paper/2108.13759/) (2021)
- [A Fine-grained Interpretability Evaluation Benchmark for Neural NLP](https://mlsanity.com/paper/2205.11097/) (2022)
- [Incorporating Attribution Importance for Improving Faithfulness Metrics](https://mlsanity.com/paper/2305.10496/) (2023)
- [InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification](https://mlsanity.com/paper/2401.16475/) (2024)
- [Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU](https://mlsanity.com/paper/2109.08259/) (2021)
- [RedHOT: A Corpus of Annotated Medical Questions, Experiences, and Claims on Social Media](https://mlsanity.com/paper/2210.06331/) (2022)
- [Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction](https://mlsanity.com/paper/2112.10424/) (2021)
- [Flexible Instance-Specific Rationalization of NLP Models](https://mlsanity.com/paper/2104.08219/) (2021)
- [Jointly Extracting Interventions, Outcomes, and Findings from RCT Reports with LLMs](https://mlsanity.com/paper/2305.03642/) (2023)
- [Goodhart's Law Applies to NLP's Explanation Benchmarks](https://mlsanity.com/paper/2308.14272/) (2023)
- [An Information Retrieval Approach to Building Datasets for Hate Speech Detection](https://mlsanity.com/paper/2106.09775/) (2021)
- [On Evaluating Explanation Utility for Human-AI Decision Making in NLP](https://mlsanity.com/paper/2407.03545/) (2024)
- [DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections](https://mlsanity.com/paper/2508.12116/) (2025)
- [Making Attention Mechanisms More Robust and Interpretable with Virtual Adversarial Training](https://mlsanity.com/paper/2104.08763/) (2021)

## Abstract (arXiv)

How do we know if a particular medical treatment actually works? Ideally one would consult all available evidence from relevant clinical trials. Unfortunately, such results are primarily disseminated in natural language scientific articles, imposing substantial burden on those trying to make sense of them. In this paper, we present a new task and corpus for making this unstructured evidence actionable. The task entails inferring reported findings from a full-text article describing a randomized controlled trial (RCT) with respect to a given intervention, comparator, and outcome of interest, e.g., inferring if an article provides evidence supporting the use of aspirin to reduce risk of stroke, as compared to placebo. We present a new corpus for this task comprising 10,000+ prompts coupled with full-text articles describing RCTs. Results using a suite of models --- ranging from heuristic (rule-based) approaches to attentive neural architectures --- demonstrate the difficulty of the task, which we believe largely owes to the lengthy, technical input texts. To facilitate further work on this important, challenging problem we make the corpus, documentation, a website and leaderboard, and code for baselines and evaluation available at http://evidence-inference.ebm-nlp.com/.