# Causality extraction from medical text using Large Language Models (LLMs)

arXiv: https://arxiv.org/abs/2407.10020 · HTML: https://mlsanity.com/paper/2407.10020/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 73 references that are in the map.
- First posted on arXiv 13 Jul 2024.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [Does Synthetic Data Generation of LLMs Help Clinical Text Mining?](https://mlsanity.com/paper/2303.04360/) (2023)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2108.13161/) (2021)
- [e-CARE: a New Dataset for Exploring Explainable Causal Reasoning](https://mlsanity.com/paper/2205.05849/) (2022)
- [ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations](https://mlsanity.com/paper/2304.14827/) (2023)
- [Causality Extraction based on Self-Attentive BiLSTM-CRF with Transferred Embeddings](https://mlsanity.com/paper/1904.07629/) (2019)
- [Causal BERT : Language models for causality detection between events expressed in text](https://mlsanity.com/paper/2012.05453/) (2020)
- [HeadlineCause: A Dataset of News Headlines for Detecting Causalities](https://mlsanity.com/paper/2108.12626/) (2021)
- [Identifying Condition-Action Statements in Medical Guidelines Using Domain-Independent Features](https://mlsanity.com/paper/1706.04206/) (2017)
- [Computing Conceptual Distances between Breast Cancer Screening Guidelines: An Implementation of a Near-Peer Epistemic Model of Medical Disagreement](https://mlsanity.com/paper/2007.00709/) (2020)

## Built on by (most cited first, 7 of 7)

- [A Survey on Enhancing Causal Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2503.09326/) (2025)
- [Counterfactual Causal Inference in Natural Language with Large Language Models](https://mlsanity.com/paper/2410.06392/) (2024)
- [A Survey of Event Causality Identification: Taxonomy, Challenges, Assessment, and Prospects](https://mlsanity.com/paper/2411.10371/) (2024)
- [Refine Medical Diagnosis Using Generation Augmented Retrieval and Clinical Practice Guidelines](https://mlsanity.com/paper/2506.21615/) (2025)
- [Benchmarking LLMs for Pairwise Causal Discovery in Biomedical and Multi-Domain Contexts](https://mlsanity.com/paper/2601.15479/) (2026)
- [Quantifying Symptom Causality in Clinical Decision Making: An Exploration Using CausaLM](https://mlsanity.com/paper/2503.19394/) (2025)
- [Zero-Shot Event Causality Identification via Multi-source Evidence Fuzzy Aggregation with Large Language Models](https://mlsanity.com/paper/2506.05675/) (2025)

## Abstract (arXiv)

This study explores the potential of natural language models, including large language models, to extract causal relations from medical texts, specifically from Clinical Practice Guidelines (CPGs). The outcomes causality extraction from Clinical Practice Guidelines for gestational diabetes are presented, marking a first in the field. We report on a set of experiments using variants of BERT (BioBERT, DistilBERT, and BERT) and using Large Language Models (LLMs), namely GPT-4 and LLAMA2. Our experiments show that BioBERT performed better than other models, including the Large Language Models, with an average F1-score of 0.72. GPT-4 and LLAMA2 results show similar performance but less consistency. We also release the code and an annotated a corpus of causal statements within the Clinical Practice Guidelines for gestational diabetes.