mlsanityData to 14 Oct 2026

ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Self-Refinement

arXiv 2401.13218 ↗First posted 24 Jan 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Structural extraction of events within discourse is critical since it avails a deeper understanding of communication patterns and behavior trends. Event argument extraction (EAE), at the core of event-centric understanding, is the task of identifying role-specific text spans (i.e., arguments) for a given event. Document-level EAE (DocEAE) focuses on arguments that are scattered across an entire document. In this work, we explore open-source Large Language Models (LLMs) for DocEAE, and propose ULTRA, a hierarchical framework that extracts event arguments more cost-effectively. Further, it alleviates the positional bias issue intrinsic to LLMs. ULTRA sequentially reads text chunks of a document to generate a candidate argument set, upon which non-pertinent candidates are dropped through self-refinement. We introduce LEAFER to address the challenge LLMs face in locating the exact boundary of an argument. ULTRA outperforms strong baselines, including strong supervised models and ChatGPT, by 9.8% when evaluated by Exact Match (EM).

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 50 of its 90 references that are in the map. First posted on arXiv 24 Jan 2024.

Builds on

50 of 90 references.

References · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2017Deep reinforcement learning from human preferencescited by 3,912
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  10. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  11. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  12. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
38 more
  1. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  2. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  3. 2019How to Fine-Tune BERT for Text Classification?cited by 411
  4. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 309
  5. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  6. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  7. 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
  8. 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
  9. 2020Event Extraction by Answering (Almost) Natural Questionscited by 177
  10. 2021Document-Level Event Argument Extraction by Conditional Generationcited by 155
  11. 2023Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulnesscited by 130
  12. 2021Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extractioncited by 129
  13. 2019Hierarchical Transformers for Long Document Classificationcited by 104
  14. 2020Look at the First Sentence: Position Bias in Question Answeringcited by 84
  15. 2021DEGREE: A Data-Efficient Generation-Based Event Extraction Modelcited by 83
  16. 2019Multi-Sentence Argument Linkingcited by 81
  17. 2023Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Methodcited by 76
  18. 2023Empirical Study of Zero-Shot NER with ChatGPTcited by 60
  19. 2019Doc2EDAG: An End-to-End Document-level Framework for Chinese Financial Event Extractioncited by 54
  20. 2023Zero-shot Temporal Relation Extraction with ChatGPTcited by 53
  21. 2022POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detectioncited by 51
  22. 2023Trapping LLM Hallucinations Using Tagged Context Promptscited by 34
  23. 2021Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Trackercited by 29
  24. 2020Document-Level Event Role Filler Extraction using Multi-Granularity Contextualized Encodingcited by 28
  25. 2023A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Taskscited by 25
  26. 2022MEE: A Novel Multilingual Event Extraction Datasetcited by 17
  27. 2023A Survey on Event-based News Narrative Extractioncited by 16
  28. 2019Giveme5W1H: A Universal System for Extracting Main Events from News Articlescited by 15
  29. 2022A Neural Pairwise Ranking Model for Readability Assessmentcited by 14
  30. 2020COUGH: A Challenge Dataset and Models for COVID-19 FAQ Retrievalcited by 10
  31. 2022Few-Shot Document-Level Event Argument Extractioncited by 9
  32. 2023MOKA: Moral Knowledge Augmentation for Moral Event Extractioncited by 7
  33. 2022Generative Entity-to-Entity Stance Detection with Knowledge Graph Augmentationcited by 6
  34. 2023MAILEX: Email Event and Argument Extractioncited by 6
  35. 2023Benchmarking Large Language Models with Augmented Instructions for Fine-grained Information Extractioncited by 6
  36. 2023Characterizing Information Seeking Events in Health-Related Social Discoursecited by 5
  37. 2021Event Argument Extraction using Causal Knowledge Structurescited by 2
  38. 2023All Things Considered: Detecting Partisan Events from News Media with Cross-Article Comparisoncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 3.

12 papers.

  1. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 86
  2. 2024Beyond Single-Event Extraction: Towards Efficient Document-Level Multi-Event Argument Extractioncited by 12
  3. 2024Explicit, Implicit, and Scattered: Revisiting Event Extraction to Capture Complex Argumentscited by 9
  4. 2024Decompose, Enrich, and Extract! Schema-aware Event Extraction using LLMscited by 6
  5. 2025Event Extraction in Large Language Modelcited by 5
  6. 2025REGen: A Reliable Evaluation Framework for Generative Event Argument Extractioncited by 4
  7. 2024Document-Level Event Extraction with Definition-Driven ICLcited by 2
  8. 2025Large Language Models Are Effective Human Annotation Assistants, But Not Good Independent Annotatorscited by 2
  9. 2026Learning to Generate and Extract: A Multi-Agent Collaboration Framework For Zero-shot Document-level Event Arguments Extractioncited by 2
  10. 2026TSUBASA: Improving Long-Horizon Personalization via Evolving Memory and Self-Learning with Context Distillationcited by 1
  11. 2026Improving Information Extraction with Learned Queriescited by 1
  12. 2025SCIR: A Self-Correcting Iterative Refinement Framework for Enhanced Information Extraction Based on Schemacited by 0