# WikiCausal: Corpus and Evaluation Framework for Causal Knowledge Graph Construction

arXiv: https://arxiv.org/abs/2409.00331 · HTML: https://mlsanity.com/paper/2409.00331/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 31 Aug 2024.

## Builds on (most cited first, 15 of 15)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [Scalable Zero-shot Entity Linking with Dense Entity Retrieval](https://mlsanity.com/paper/1911.03814/) (2019)
- [Financial Document Causality Detection Shared Task (FinCausal 2020)](https://mlsanity.com/paper/2012.02505/) (2020)
- [CausalKG: Causal Knowledge Graph Explainability using interventional and counterfactual reasoning](https://mlsanity.com/paper/2201.03647/) (2022)
- [Event Linking: Grounding Event Mentions to Wikipedia](https://mlsanity.com/paper/2112.07888/) (2021)
- [UniCausal: Unified Benchmark and Repository for Causal Text Mining](https://mlsanity.com/paper/2208.09163/) (2022)
- [EMBERS at 4 years: Experiences operating an Open Source Indicators Forecasting System](https://mlsanity.com/paper/1604.00033/) (2016)
- [Event Causality Identification with Causal News Corpus -- Shared Task 3, CASE 2022](https://mlsanity.com/paper/2211.12154/) (2022)
- [Summary Markov Models for Event Sequences](https://mlsanity.com/paper/2205.03375/) (2022)
- [Event Prediction using Case-Based Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2309.12423/) (2023)

## Abstract (arXiv)

Recently, there has been an increasing interest in the construction of general-domain and domain-specific causal knowledge graphs. Such knowledge graphs enable reasoning for causal analysis and event prediction, and so have a range of applications across different domains. While great progress has been made toward automated construction of causal knowledge graphs, the evaluation of such solutions has either focused on low-level tasks (e.g., cause-effect phrase extraction) or on ad hoc evaluation data and small manual evaluations. In this paper, we present a corpus, task, and evaluation framework for causal knowledge graph construction. Our corpus consists of Wikipedia articles for a collection of event-related concepts in Wikidata. The task is to extract causal relations between event concepts from the corpus. The evaluation is performed in part using existing causal relations in Wikidata to measure recall, and in part using Large Language Models to avoid the need for manual or crowd-sourced evaluation. We evaluate a pipeline for causal knowledge graph construction that relies on neural models for question answering and concept linking, and show how the corpus and the evaluation framework allow us to effectively find the right model for each task. The corpus and the evaluation framework are publicly available.