# TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction

arXiv: https://arxiv.org/abs/2311.09562 · HTML: https://mlsanity.com/paper/2311.09562/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 88 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Builds on (most cited first, 30 of 57)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [Unified Structure Generation for Universal Information Extraction](https://mlsanity.com/paper/2203.12277/) (2022)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Event Extraction by Answering (Almost) Natural Questions](https://mlsanity.com/paper/2004.13625/) (2020)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [Document-Level Event Argument Extraction by Conditional Generation](https://mlsanity.com/paper/2104.05919/) (2021)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction](https://mlsanity.com/paper/2106.09232/) (2021)
- [MAVEN: A Massive General Domain Event Detection Dataset](https://mlsanity.com/paper/2004.13590/) (2020)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [ASER: A Large-scale Eventuality Knowledge Graph](https://mlsanity.com/paper/1905.00270/) (2019)
- [CLIP-Event: Connecting Text and Images with Event Structures](https://mlsanity.com/paper/2201.05078/) (2022)
- [DEGREE: A Data-Efficient Generation-Based Event Extraction Model](https://mlsanity.com/paper/2108.12724/) (2021)
- [Multi-Sentence Argument Linking](https://mlsanity.com/paper/1911.03766/) (2019)
- [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023)
- [Code4Struct: Code Generation for Few-Shot Event Structure Prediction](https://mlsanity.com/paper/2210.12810/) (2022)
- [Zero-Shot Transfer Learning for Event Extraction](https://mlsanity.com/paper/1707.01066/) (2017)
- [Cross-media Structured Common Space for Multimedia Event Extraction](https://mlsanity.com/paper/2005.02472/) (2020)
- [Meta-Learning with Dynamic-Memory-Based Prototypical Network for Few-Shot Event Detection](https://mlsanity.com/paper/1910.11621/) (2019)
- [Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction](https://mlsanity.com/paper/2202.12109/) (2022)
- [Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction](https://mlsanity.com/paper/1909.05360/) (2019)
- [Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction](https://mlsanity.com/paper/2203.08308/) (2022)

## Built on by (most cited first, 28 of 28)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Instruction-Tuning LLMs for Event Extraction with Annotation Guidelines](https://mlsanity.com/paper/2502.16377/) (2025)
- [Explicit, Implicit, and Scattered: Revisiting Event Extraction to Capture Complex Arguments](https://mlsanity.com/paper/2410.03594/) (2024)
- [SPEED++: A Multilingual Event Extraction Framework for Epidemic Prediction and Preparedness](https://mlsanity.com/paper/2410.18393/) (2024)
- [MMUTF: Multimodal Multimedia Event Argument Extraction with Unified Template Filling](https://mlsanity.com/paper/2406.12420/) (2024)
- [Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extraction](https://mlsanity.com/paper/2504.07357/) (2025)
- [Event Extraction in Large Language Model](https://mlsanity.com/paper/2512.19537/) (2025)
- [REGen: A Reliable Evaluation Framework for Generative Event Argument Extraction](https://mlsanity.com/paper/2502.16838/) (2025)
- [SNaRe: Domain-aware Data Generation for Low-Resource Event Detection](https://mlsanity.com/paper/2502.17394/) (2025)
- [SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection](https://mlsanity.com/paper/2503.03303/) (2025)
- [A Structure-aware Generative Model for Biomedical Event Extraction](https://mlsanity.com/paper/2408.06583/) (2024)
- [LEMONADE: A Large Multilingual Expert-Annotated Abstractive Event Dataset for the Real World](https://mlsanity.com/paper/2506.00980/) (2025)
- [EXCEEDS: Extracting Complex Events via Nugget-based Grid Modeling in Scientific Domain](https://mlsanity.com/paper/2406.14075/) (2024)
- [Enhancing Event Reasoning in Large Language Models through Instruction Fine-Tuning with Semantic Causal Graphs](https://mlsanity.com/paper/2409.00209/) (2024)
- [One Small and One Large for Document-level Event Argument Extraction](https://mlsanity.com/paper/2411.05895/) (2024)
- [SciEvent: Benchmarking Multi-domain Scientific Event Extraction](https://mlsanity.com/paper/2509.15620/) (2025)
- [Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes](https://mlsanity.com/paper/2601.11932/) (2026)
- [Improving Information Extraction with Learned Queries](https://mlsanity.com/paper/2608.31058/) (2026)
- [Beyond Exact Match: Semantically Reassessing Event Extraction by Large Language Models](https://mlsanity.com/paper/2410.09418/) (2024)
- [Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction](https://mlsanity.com/paper/2508.19359/) (2025)
- [Event-Centric Human Value Understanding in News-Domain Texts: An Actor-Conditioned, Multi-Granularity Benchmark](https://mlsanity.com/paper/2603.17838/) (2026)
- [Evaluation Pitfalls and Challenges in Multimedia Event Extraction](https://mlsanity.com/paper/2606.26775/) (2026)
- [From Single- to Cross-Document: Benchmarking Multi-Granularity Event Analysis of Large Language Models](https://mlsanity.com/paper/2607.27654/) (2026)
- [A Multi-Domain and Multi-Task Generative Framework with Explicit Task and Domain Conditioning for Cross-Domain Event Extraction](https://mlsanity.com/paper/2608.23235/) (2026)
- [A Scalable Cross-Domain Event Extraction System via a Unified Generative Training Framework](https://mlsanity.com/paper/2608.23261/) (2026)
- [CMNIE: An Information Extraction Benchmark for Chinese Military News](https://mlsanity.com/paper/2609.10722/) (2026)
- [EAGER: Enhancing Generative Event Extraction via Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2609.29230/) (2026)
- [CLUE: A Contrastive Learning Framework for Unsupervised Event Type Induction and Annotation](https://mlsanity.com/paper/2610.12580/) (2026)

## Abstract (arXiv)

Event extraction has gained considerable interest due to its wide-ranging applications. However, recent studies draw attention to evaluation issues, suggesting that reported scores may not accurately reflect the true performance. In this work, we identify and address evaluation challenges, including inconsistency due to varying data assumptions or preprocessing steps, the insufficiency of current evaluation frameworks that may introduce dataset or data split bias, and the low reproducibility of some previous approaches. To address these challenges, we present TextEE, a standardized, fair, and reproducible benchmark for event extraction. TextEE comprises standardized data preprocessing scripts and splits for 16 datasets spanning eight diverse domains and includes 14 recent methodologies, conducting a comprehensive benchmark reevaluation. We also evaluate five varied large language models on our TextEE benchmark and demonstrate how they struggle to achieve satisfactory performance. Inspired by our reevaluation results and findings, we discuss the role of event extraction in the current NLP era, as well as future challenges and insights derived from TextEE. We believe TextEE, the first standardized comprehensive benchmarking tool, will significantly facilitate future event extraction research.