# MAVEN-Fact: A Large-scale Event Factuality Detection Dataset

arXiv: https://arxiv.org/abs/2407.15352 · HTML: https://mlsanity.com/paper/2407.15352/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 54 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [MAVEN: A Massive General Domain Event Detection Dataset](https://mlsanity.com/paper/2004.13590/) (2020)
- [Towards Mitigating Hallucination in Large Language Models via Self-Reflection](https://mlsanity.com/paper/2310.06271/) (2023)
- [Code4Struct: Code Generation for Few-Shot Event Structure Prediction](https://mlsanity.com/paper/2210.12810/) (2022)
- [MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction](https://mlsanity.com/paper/2211.07342/) (2022)
- [Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations](https://mlsanity.com/paper/2305.13299/) (2023)
- [Plan, Write, and Revise: an Interactive System for Open-Domain Story Generation](https://mlsanity.com/paper/1904.02357/) (2019)
- [When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks](https://mlsanity.com/paper/2311.08993/) (2023)
- [Are Large Language Models Reliable Argument Quality Annotators?](https://mlsanity.com/paper/2404.09696/) (2024)
- [ADELIE: Aligning Large Language Models on Information Extraction](https://mlsanity.com/paper/2405.05008/) (2024)
- [Neural models of factuality](https://mlsanity.com/paper/1804.02472/) (2018)
- [The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation](https://mlsanity.com/paper/2306.06918/) (2023)
- [Graph based Neural Networks for Event Factuality Prediction using Syntactic and Semantic Structures](https://mlsanity.com/paper/1907.03227/) (2019)
- [Sequence Labeling: A Practical Approach](https://mlsanity.com/paper/1808.03926/) (2018)

## Built on by (most cited first, 6 of 6)

- [Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks](https://mlsanity.com/paper/2504.18838/) (2025)
- [Emergence of Linear Truth Encodings in Language Models](https://mlsanity.com/paper/2510.15804/) (2025)
- [WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Quality](https://mlsanity.com/paper/2510.18560/) (2025)
- [A Multi-Level Benchmark for Causal Language Understanding in Social Media Discourse](https://mlsanity.com/paper/2509.16722/) (2025)
- [Zero-Shot Belief: A Hard Problem for LLMs](https://mlsanity.com/paper/2502.08777/) (2025)
- [An Annotation Scheme for Factuality and its Application to Parliamentary Proceedings](https://mlsanity.com/paper/2509.26406/) (2025)

## Abstract (arXiv)

Event Factuality Detection (EFD) task determines the factuality of textual events, i.e., classifying whether an event is a fact, possibility, or impossibility, which is essential for faithfully understanding and utilizing event knowledge. However, due to the lack of high-quality large-scale data, event factuality detection is under-explored in event understanding research, which limits the development of EFD community. To address these issues and provide faithful event understanding, we introduce MAVEN-Fact, a large-scale and high-quality EFD dataset based on the MAVEN dataset. MAVEN-Fact includes factuality annotations of 112,276 events, making it the largest EFD dataset. Extensive experiments demonstrate that MAVEN-Fact is challenging for both conventional fine-tuned models and large language models (LLMs). Thanks to the comprehensive annotations of event arguments and relations in MAVEN, MAVEN-Fact also supports some further analyses and we find that adopting event arguments and relations helps in event factuality detection for fine-tuned models but does not benefit LLMs. Furthermore, we preliminarily study an application case of event factuality detection and find it helps in mitigating event-related hallucination in LLMs. Our dataset and codes can be obtained from urlhttps://github.com/lcy2723/MAVEN-FACT