# Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes

arXiv: https://arxiv.org/abs/2601.11932 · HTML: https://mlsanity.com/paper/2601.11932/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 17 Jan 2026.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [MAVEN: A Massive General Domain Event Detection Dataset](https://mlsanity.com/paper/2004.13590/) (2020)
- [ASER: A Large-scale Eventuality Knowledge Graph](https://mlsanity.com/paper/1905.00270/) (2019)
- [Multi-Sentence Argument Linking](https://mlsanity.com/paper/1911.03766/) (2019)
- [Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks](https://mlsanity.com/paper/2406.13469/) (2024)
- [ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning](https://mlsanity.com/paper/2104.08350/) (2021)
- [TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction](https://mlsanity.com/paper/2311.09562/) (2023)
- [Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling](https://mlsanity.com/paper/2401.14556/) (2024)
- [Continual Event Extraction with Semantic Confusion Rectification](https://mlsanity.com/paper/2310.15470/) (2023)

## Built on by (most cited first, 1 of 1)

- [Beyond Clean Text: Evaluating Encoder and Decoder Robustness for Bangla Event Detection in Noisy Text](https://mlsanity.com/paper/2606.30914/) (2026)

## Abstract (arXiv)

The current state of event detection research has two notable re-occurring limitations that we investigate in this study. First, the unidirectional nature of decoder-only LLMs presents a fundamental architectural bottleneck for natural language understanding tasks that depend on rich, bidirectional context. Second, we confront the conventional reliance on Micro-F1 scores in event detection literature, which systematically inflates performance by favoring majority classes. Instead, we focus on Macro-F1 as a more representative measure of a model's ability across the long-tail of event types. Our experiments demonstrate that models enhanced with sentence context achieve superior performance over canonical decoder-only baselines. Using Low-Rank Adaptation (LoRA) during finetuning provides a substantial boost in Macro-F1 scores in particular, especially for the decoder-only models, showing that LoRA can be an effective tool to enhance LLMs' performance on long-tailed event classes.