# A Survey on Extraction of Causal Relations from Natural Language Text

arXiv: https://arxiv.org/abs/2101.06426 · HTML: https://mlsanity.com/paper/2101.06426/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 123 references that are in the map.
- First posted on arXiv 16 Jan 2021.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019)
- [Graph Convolution over Pruned Dependency Trees Improves Relation Extraction](https://mlsanity.com/paper/1809.10185/) (2018)
- [Double Graph Based Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2009.13752/) (2020)
- [Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders](https://mlsanity.com/paper/2010.03851/) (2020)
- [Attention Guided Graph Convolutional Networks for Relation Extraction](https://mlsanity.com/paper/1906.07510/) (2019)
- [Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs](https://mlsanity.com/paper/1909.00228/) (2019)
- [SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals](https://mlsanity.com/paper/1911.10422/) (2019)
- [Joint entity recognition and relation extraction as a multi-head selection problem](https://mlsanity.com/paper/1804.07847/) (2018)
- [Global-to-Local Neural Networks for Document-Level Relation Extraction](https://mlsanity.com/paper/2009.10359/) (2020)
- [Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path](https://mlsanity.com/paper/1508.03720/) (2015)
- [Adversarial training for multi-context joint entity and relation extraction](https://mlsanity.com/paper/1808.06876/) (2018)
- [Causality Extraction based on Self-Attentive BiLSTM-CRF with Transferred Embeddings](https://mlsanity.com/paper/1904.07629/) (2019)
- [Improving Relation Extraction by Pre-trained Language Representations](https://mlsanity.com/paper/1906.03088/) (2019)
- [Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers](https://mlsanity.com/paper/1902.01030/) (2019)
- [Automatic Extraction of Causal Relations from Natural Language Texts: A Comprehensive Survey](https://mlsanity.com/paper/1605.07895/) (2016)
- [A Walk-based Model on Entity Graphs for Relation Extraction](https://mlsanity.com/paper/1902.07023/) (2019)
- [Transfer Learning for Causal Sentence Detection](https://mlsanity.com/paper/1906.07544/) (2019)
- [Relabel the Noise: Joint Extraction of Entities and Relations via Cooperative Multiagents](https://mlsanity.com/paper/2004.09930/) (2020)
- [A Multi-level Neural Network for Implicit Causality Detection in Web Texts](https://mlsanity.com/paper/1908.07822/) (2019)

## Built on by (most cited first, 25 of 25)

- [A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions](https://mlsanity.com/paper/2406.03712/) (2024)
- [The Causal News Corpus: Annotating Causal Relations in Event Sentences from News](https://mlsanity.com/paper/2204.11714/) (2022)
- [A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations](https://mlsanity.com/paper/2406.10303/) (2024)
- [UniCausal: Unified Benchmark and Repository for Causal Text Mining](https://mlsanity.com/paper/2208.09163/) (2022)
- [Large Causal Models from Large Language Models](https://mlsanity.com/paper/2512.07796/) (2025)
- [Leveraging Prompt-Based Large Language Models: Predicting Pandemic Health Decisions and Outcomes Through Social Media Language](https://mlsanity.com/paper/2403.00994/) (2024)
- [A Survey of Event Causality Identification: Taxonomy, Challenges, Assessment, and Prospects](https://mlsanity.com/paper/2411.10371/) (2024)
- [HeadlineCause: A Dataset of News Headlines for Detecting Causalities](https://mlsanity.com/paper/2108.12626/) (2021)
- [An Explainable AI Approach to Large Language Model Assisted Causal Model Auditing and Development](https://mlsanity.com/paper/2312.16211/) (2023)
- [Event Causality Identification with Causal News Corpus -- Shared Task 3, CASE 2022](https://mlsanity.com/paper/2211.12154/) (2022)
- [CausalChat: Interactive Causal Model Development and Refinement Using Large Language Models](https://mlsanity.com/paper/2410.14146/) (2024)
- [PROMETHEUS: Automating Deep Causal Research Integrating Text, Data and Models](https://mlsanity.com/paper/2605.12835/) (2026)
- [Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memes](https://mlsanity.com/paper/2308.11585/) (2023)
- [Is ChatGPT the Future of Causal Text Mining? A Comprehensive Evaluation and Analysis](https://mlsanity.com/paper/2402.14484/) (2024)
- [Fine-Grained Causality Extraction From Natural Language Requirements Using Recursive Neural Tensor Networks](https://mlsanity.com/paper/2107.09980/) (2021)
- [CATE: CAusality Tree Extractor from Natural Language Requirements](https://mlsanity.com/paper/2107.10023/) (2021)
- [Identifying causal relations in tweets using deep learning: Use case on diabetes-related tweets from 2017-2021](https://mlsanity.com/paper/2111.01225/) (2021)
- [Applying BioBERT to Extract Germline Gene-Disease Associations for Building a Knowledge Graph from the Biomedical Literature](https://mlsanity.com/paper/2309.13061/) (2023)
- [Causality Extraction from Nuclear Licensee Event Reports Using a Hybrid Framework](https://mlsanity.com/paper/2404.05656/) (2024)
- [Causality Elicitation from Large Language Models](https://mlsanity.com/paper/2603.04276/) (2026)
- [Odyssey: Constructing Verifiable Local Truth-Preserving Foundation Models](https://mlsanity.com/paper/2606.27593/) (2026)
- [Increasing the Accessibility of Causal Domain Knowledge via Causal Information Extraction Methods: A Case Study in the Semiconductor Manufacturing Industry](https://mlsanity.com/paper/2411.10172/) (2024)
- [Evaluating Large Language Models for Causal Modeling](https://mlsanity.com/paper/2411.15888/) (2024)
- [Causal Cartographer: From Mapping to Reasoning Over Counterfactual Worlds](https://mlsanity.com/paper/2505.14396/) (2025)
- [CSQL: Mapping Documents into Causal Databases](https://mlsanity.com/paper/2601.08109/) (2026)

## Abstract (arXiv)

As an essential component of human cognition, cause-effect relations appear frequently in text, and curating cause-effect relations from text helps in building causal networks for predictive tasks. Existing causality extraction techniques include knowledge-based, statistical machine learning(ML)-based, and deep learning-based approaches. Each method has its advantages and weaknesses. For example, knowledge-based methods are understandable but require extensive manual domain knowledge and have poor cross-domain applicability. Statistical machine learning methods are more automated because of natural language processing (NLP) toolkits. However, feature engineering is labor-intensive, and toolkits may lead to error propagation. In the past few years, deep learning techniques attract substantial attention from NLP researchers because of its' powerful representation learning ability and the rapid increase in computational resources. Their limitations include high computational costs and a lack of adequate annotated training data. In this paper, we conduct a comprehensive survey of causality extraction. We initially introduce primary forms existing in the causality extraction: explicit intra-sentential causality, implicit causality, and inter-sentential causality. Next, we list benchmark datasets and modeling assessment methods for causal relation extraction. Then, we present a structured overview of the three techniques with their representative systems. Lastly, we highlight existing open challenges with their potential directions.