# ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning

arXiv: https://arxiv.org/abs/2104.08350 · HTML: https://mlsanity.com/paper/2104.08350/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 14 of 14)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning](https://mlsanity.com/paper/1909.00277/) (2019)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Event Extraction by Answering (Almost) Natural Questions](https://mlsanity.com/paper/2004.13625/) (2020)
- [Document-Level Event Argument Extraction by Conditional Generation](https://mlsanity.com/paper/2104.05919/) (2021)
- ["Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understanding](https://mlsanity.com/paper/1909.03065/) (2019)
- [Learning Named Entity Tagger using Domain-Specific Dictionary](https://mlsanity.com/paper/1809.03599/) (2018)
- [TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions](https://mlsanity.com/paper/2005.00242/) (2020)
- [Easy, Reproducible and Quality-Controlled Data Collection with Crowdaq](https://mlsanity.com/paper/2010.06694/) (2020)
- [Learning A Unified Named Entity Tagger From Multiple Partially Annotated Corpora For Efficient Adaptation](https://mlsanity.com/paper/1909.11535/) (2019)

## Built on by (most cited first, 30 of 30)

- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [DEGREE: A Data-Efficient Generation-Based Event Extraction Model](https://mlsanity.com/paper/2108.12724/) (2021)
- [Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning](https://mlsanity.com/paper/2305.16646/) (2023)
- [TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction](https://mlsanity.com/paper/2311.09562/) (2023)
- [GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles](https://mlsanity.com/paper/2205.12505/) (2022)
- [Open Domain Question Answering with Conflicting Contexts](https://mlsanity.com/paper/2410.12311/) (2024)
- [Go Back in Time: Generating Flashbacks in Stories with Event Temporal Prompts](https://mlsanity.com/paper/2205.01898/) (2022)
- [Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning](https://mlsanity.com/paper/2305.14970/) (2023)
- [A Survey on Deep Learning Event Extraction: Approaches and Applications](https://mlsanity.com/paper/2107.02126/) (2021)
- [A Survey of Machine Narrative Reading Comprehension Assessments](https://mlsanity.com/paper/2205.00299/) (2022)
- [An AMR-based Link Prediction Approach for Document-level Event Argument Extraction](https://mlsanity.com/paper/2305.19162/) (2023)
- [CoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding](https://mlsanity.com/paper/2212.10754/) (2022)
- [EvEval: A Comprehensive Evaluation of Event Semantics for Large Language Models](https://mlsanity.com/paper/2305.15268/) (2023)
- [Decompose, Enrich, and Extract! Schema-aware Event Extraction using LLMs](https://mlsanity.com/paper/2406.01045/) (2024)
- [Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformation](https://mlsanity.com/paper/2210.12902/) (2022)
- [DemoSG: Demonstration-enhanced Schema-guided Generation for Low-resource Event Extraction](https://mlsanity.com/paper/2310.10481/) (2023)
- [Generating Uncontextualized and Contextualized Questions for Document-Level Event Argument Extraction](https://mlsanity.com/paper/2404.04770/) (2024)
- [Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention](https://mlsanity.com/paper/2310.15258/) (2023)
- [From Simple to Complex: A Progressive Framework for Document-level Informative Argument Extraction](https://mlsanity.com/paper/2310.16358/) (2023)
- [Some Reflections on Drawing Causal Inference using Textual Data: Parallels Between Human Subjects and Organized Texts](https://mlsanity.com/paper/2202.00848/) (2022)
- [Event Knowledge Incorporation with Posterior Regularization for Event-Centric Question Answering](https://mlsanity.com/paper/2305.04522/) (2023)
- [MEEL: Multi-Modal Event Evolution Learning](https://mlsanity.com/paper/2404.10429/) (2024)
- [Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes](https://mlsanity.com/paper/2601.11932/) (2026)
- [Hierarchical Event Grounding](https://mlsanity.com/paper/2302.04197/) (2023)
- [Modeling Complex Event Scenarios via Simple Entity-focused Questions](https://mlsanity.com/paper/2302.07139/) (2023)
- [Causal schema induction for knowledge discovery](https://mlsanity.com/paper/2303.15381/) (2023)
- [Type-aware Decoding via Explicitly Aggregating Event Information for Document-level Event Extraction](https://mlsanity.com/paper/2310.10487/) (2023)
- [QLSC: A Query Latent Semantic Calibrator for Robust Extractive Question Answering](https://mlsanity.com/paper/2404.19316/) (2024)
- [Beyond Exact Match: Semantically Reassessing Event Extraction by Large Language Models](https://mlsanity.com/paper/2410.09418/) (2024)
- [Keyword-Centric Prompting for One-Shot Event Detection with Self-Generated Rationale Enhancements](https://mlsanity.com/paper/2508.07598/) (2025)

## Abstract (arXiv)

Understanding how events are semantically related to each other is the essence of reading comprehension. Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. While these tasks partially evaluate machines' ability of narrative understanding, human-like reading comprehension requires the capability to process event-based information beyond arguments and temporal reasoning. For example, to understand causality between events, we need to infer motivation or purpose; to establish event hierarchy, we need to understand the composition of events. To facilitate these tasks, we introduce ESTER, a comprehensive machine reading comprehension (MRC) dataset for Event Semantic Relation Reasoning. The dataset leverages natural language queries to reason about the five most common event semantic relations, provides more than 6K questions and captures 10.1K event relation pairs. Experimental results show that the current SOTA systems achieve 22.1%, 63.3%, and 83.5% for token-based exact-match, F1, and event-based HIT@1 scores, which are all significantly below human performances (36.0%, 79.6%, 100% respectively), highlighting our dataset as a challenging benchmark.