# Instruction-Tuning LLMs for Event Extraction with Annotation Guidelines

arXiv: https://arxiv.org/abs/2502.16377 · HTML: https://mlsanity.com/paper/2502.16377/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 22 Feb 2025.

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA](https://mlsanity.com/paper/2312.03732/) (2023)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction](https://mlsanity.com/paper/2304.08085/) (2023)
- [Document-Level Event Argument Extraction by Conditional Generation](https://mlsanity.com/paper/2104.05919/) (2021)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [Exploring the Feasibility of ChatGPT for Event Extraction](https://mlsanity.com/paper/2303.03836/) (2023)
- [Code4Struct: Code Generation for Few-Shot Event Structure Prediction](https://mlsanity.com/paper/2210.12810/) (2022)
- [CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors](https://mlsanity.com/paper/2305.05711/) (2023)
- [Retrieval-Augmented Code Generation for Universal Information Extraction](https://mlsanity.com/paper/2311.02962/) (2023)
- [Instruct and Extract: Instruction Tuning for On-Demand Information Extraction](https://mlsanity.com/paper/2310.16040/) (2023)
- [KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction](https://mlsanity.com/paper/2403.07969/) (2024)
- [TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction](https://mlsanity.com/paper/2311.09562/) (2023)
- [Guideline Learning for In-context Information Extraction](https://mlsanity.com/paper/2310.05066/) (2023)
- [Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner](https://mlsanity.com/paper/2305.01711/) (2023)
- [InstructIE: A Bilingual Instruction-based Information Extraction Dataset](https://mlsanity.com/paper/2305.11527/) (2023)
- [Improving Event Definition Following For Zero-Shot Event Detection](https://mlsanity.com/paper/2403.02586/) (2024)
- [Are LLMs Good Annotators for Discourse-level Event Relation Extraction?](https://mlsanity.com/paper/2407.19568/) (2024)
- [SPEED++: A Multilingual Event Extraction Framework for Epidemic Prediction and Preparedness](https://mlsanity.com/paper/2410.18393/) (2024)
- [MAILEX: Email Event and Argument Extraction](https://mlsanity.com/paper/2305.13469/) (2023)

## Built on by (most cited first, 18 of 18)

- [Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extraction](https://mlsanity.com/paper/2504.07357/) (2025)
- [Event Extraction in Large Language Model](https://mlsanity.com/paper/2512.19537/) (2025)
- [DiCoRe: Enhancing Zero-shot Event Detection via Divergent-Convergent LLM Reasoning](https://mlsanity.com/paper/2506.05128/) (2025)
- [Extracting Events Like Code: A Multi-Agent Programming Framework for Zero-Shot Event Extraction](https://mlsanity.com/paper/2511.13118/) (2025)
- [Multimodal Event Detection: Current Approaches and Defining the New Playground through LLMs and VLMs](https://mlsanity.com/paper/2505.10836/) (2025)
- [When Does Meaning Backfire? Investigating the Role of AMRs in NLI](https://mlsanity.com/paper/2506.14613/) (2025)
- [Batch Prompting Suppresses Overthinking Reasoning Under Constraint: How Batch Prompting Suppresses Overthinking in Reasoning Models](https://mlsanity.com/paper/2511.04108/) (2025)
- [Improving Information Extraction with Learned Queries](https://mlsanity.com/paper/2608.31058/) (2026)
- [Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction](https://mlsanity.com/paper/2508.19359/) (2025)
- [ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference](https://mlsanity.com/paper/2510.02361/) (2025)
- [RMPL: Relation-aware Multi-task Progressive Learning with Stage-wise Training for Multimedia Event Extraction](https://mlsanity.com/paper/2602.13748/) (2026)
- [Refining and Reusing Annotation Guidelines for LLM Annotation](https://mlsanity.com/paper/2605.20809/) (2026)
- [SL-BiLEM: Structured Learnable Behavior-in-the-Loop Epidemic Modeling for Forecasting and Policy Evaluation](https://mlsanity.com/paper/2605.26704/) (2026)
- [Applicability Condition Extraction for Therapeutic Drug-Disease Relations](https://mlsanity.com/paper/2606.14031/) (2026)
- [Beyond Clean Text: Evaluating Encoder and Decoder Robustness for Bangla Event Detection in Noisy Text](https://mlsanity.com/paper/2606.30914/) (2026)
- [A Multi-Domain and Multi-Task Generative Framework with Explicit Task and Domain Conditioning for Cross-Domain Event Extraction](https://mlsanity.com/paper/2608.23235/) (2026)
- [EAGER: Enhancing Generative Event Extraction via Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2609.29230/) (2026)
- [ARGUS: Role-Aware Event Knowledge Graphs for U.S. Employment-Discrimination Complaints](https://mlsanity.com/paper/2609.30184/) (2026)

## Abstract (arXiv)

In this work, we study the effect of annotation guidelines -- textual descriptions of event types and arguments, when instruction-tuning large language models for event extraction. We conducted a series of experiments with both human-provided and machine-generated guidelines in both full- and low-data settings. Our results demonstrate the promise of annotation guidelines when there is a decent amount of training data and highlight its effectiveness in improving cross-schema generalization and low-frequency event-type performance.