# Language Model Priming for Cross-Lingual Event Extraction

arXiv: https://arxiv.org/abs/2109.12383 · HTML: https://mlsanity.com/paper/2109.12383/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 25 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [How Many Data Points is a Prompt Worth?](https://mlsanity.com/paper/2103.08493/) (2021)
- [Event Extraction by Answering (Almost) Natural Questions](https://mlsanity.com/paper/2004.13625/) (2020)
- [Entity-Relation Extraction as Multi-Turn Question Answering](https://mlsanity.com/paper/1905.05529/) (2019)
- [Coreference Resolution as Query-based Span Prediction](https://mlsanity.com/paper/1911.01746/) (2019)
- [Joint Extraction of Events and Entities within a Document Context](https://mlsanity.com/paper/1609.03632/) (2016)
- [GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction](https://mlsanity.com/paper/2010.03009/) (2020)
- [Gradual Fine-Tuning for Low-Resource Domain Adaptation](https://mlsanity.com/paper/2103.02205/) (2021)
- [Probing and Fine-tuning Reading Comprehension Models for Few-shot Event Extraction](https://mlsanity.com/paper/2010.11325/) (2020)

## Built on by (most cited first, 13 of 13)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction](https://mlsanity.com/paper/2203.08308/) (2022)
- [AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model](https://mlsanity.com/paper/2305.16734/) (2023)
- [Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games](https://mlsanity.com/paper/2310.00322/) (2023)
- [TAGPRIME: A Unified Framework for Relational Structure Extraction](https://mlsanity.com/paper/2205.12585/) (2022)
- [Information Extraction in Low-Resource Scenarios: Survey and Perspective](https://mlsanity.com/paper/2202.08063/) (2022)
- [On the Evolution of Knowledge Graphs: A Survey and Perspective](https://mlsanity.com/paper/2310.04835/) (2023)
- [Contextual Label Projection for Cross-Lingual Structured Prediction](https://mlsanity.com/paper/2309.08943/) (2023)
- [Massively Multi-Lingual Event Understanding: Extraction, Visualization, and Search](https://mlsanity.com/paper/2305.10561/) (2023)
- [Event Extraction in Large Language Model](https://mlsanity.com/paper/2512.19537/) (2025)
- [MultiMUC: Multilingual Template Filling on MUC-4](https://mlsanity.com/paper/2401.16209/) (2024)
- [Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification](https://mlsanity.com/paper/2308.07876/) (2023)
- [Impact of Subword Pooling Strategy on Cross-lingual Event Detection](https://mlsanity.com/paper/2302.11365/) (2023)

## Abstract (arXiv)

We present a novel, language-agnostic approach to "priming" language models for the task of event extraction, providing particularly effective performance in low-resource and zero-shot cross-lingual settings. With priming, we augment the input to the transformer stack's language model differently depending on the question(s) being asked of the model at runtime. For instance, if the model is being asked to identify arguments for the trigger "protested", we will provide that trigger as part of the input to the language model, allowing it to produce different representations for candidate arguments than when it is asked about arguments for the trigger "arrest" elsewhere in the same sentence. We show that by enabling the language model to better compensate for the deficits of sparse and noisy training data, our approach improves both trigger and argument detection and classification significantly over the state of the art in a zero-shot cross-lingual setting.