# GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction

arXiv: https://arxiv.org/abs/2010.03009 · HTML: https://mlsanity.com/paper/2010.03009/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 30 of 43)

- [Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction](https://mlsanity.com/paper/2203.08308/) (2022)
- [Frustratingly Easy Label Projection for Cross-lingual Transfer](https://mlsanity.com/paper/2211.15613/) (2022)
- [Graph Transformers: A Survey](https://mlsanity.com/paper/2407.09777/) (2024)
- [Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training](https://mlsanity.com/paper/2104.08645/) (2021)
- [mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2110.08151/) (2021)
- [AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model](https://mlsanity.com/paper/2305.16734/) (2023)
- [Breaking the Entanglement of Homophily and Heterophily in Semi-supervised Node Classification](https://mlsanity.com/paper/2312.04111/) (2023)
- [EventPlus: A Temporal Event Understanding Pipeline](https://mlsanity.com/paper/2101.04922/) (2021)
- [MEE: A Novel Multilingual Event Extraction Dataset](https://mlsanity.com/paper/2211.05955/) (2022)
- [SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning](https://mlsanity.com/paper/2112.08587/) (2021)
- [CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval](https://mlsanity.com/paper/2209.02071/) (2022)
- [Language Model Priming for Cross-Lingual Event Extraction](https://mlsanity.com/paper/2109.12383/) (2021)
- [A Survey on Deep Learning Event Extraction: Approaches and Applications](https://mlsanity.com/paper/2107.02126/) (2021)
- [Syntax-augmented Multilingual BERT for Cross-lingual Transfer](https://mlsanity.com/paper/2106.02134/) (2021)
- [Improved Beam Search for Hallucination Mitigation in Abstractive Summarization](https://mlsanity.com/paper/2212.02712/) (2022)
- [A Hierarchical N-Gram Framework for Zero-Shot Link Prediction](https://mlsanity.com/paper/2204.10293/) (2022)
- [Contextual Label Projection for Cross-Lingual Structured Prediction](https://mlsanity.com/paper/2309.08943/) (2023)
- [MINION: a Large-Scale and Diverse Dataset for Multilingual Event Detection](https://mlsanity.com/paper/2211.05958/) (2022)
- [Syntax and Domain Aware Model for Unsupervised Program Translation](https://mlsanity.com/paper/2302.03908/) (2023)
- [Controllable Text Generation for Open-Domain Creativity and Fairness](https://mlsanity.com/paper/2209.12099/) (2022)
- [Constructing Code-mixed Universal Dependency Forest for Unbiased Cross-lingual Relation Extraction](https://mlsanity.com/paper/2305.12258/) (2023)
- [Event Extraction by Associating Event Types and Argument Roles](https://mlsanity.com/paper/2108.10038/) (2021)
- [An overview of event extraction and its applications](https://mlsanity.com/paper/2111.03212/) (2021)
- [Event Extraction in Large Language Model](https://mlsanity.com/paper/2512.19537/) (2025)
- [A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning](https://mlsanity.com/paper/2210.09934/) (2022)
- [Prompt-Learning for Cross-Lingual Relation Extraction](https://mlsanity.com/paper/2304.10354/) (2023)
- [Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation](https://mlsanity.com/paper/2311.15211/) (2023)
- [Identifying and Investigating Global News Coverage of Critical Events Such as Disasters and Terrorist Attacks](https://mlsanity.com/paper/2506.12925/) (2025)
- [Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer](https://mlsanity.com/paper/2309.10891/) (2023)
- [DEGAP: Dual Event-Guided Adaptive Prefixes for Templated-Based Event Argument Extraction with Slot Querying](https://mlsanity.com/paper/2405.13325/) (2024)

## Abstract (arXiv)

Recent progress in cross-lingual relation and event extraction use graph convolutional networks (GCNs) with universal dependency parses to learn language-agnostic sentence representations such that models trained on one language can be applied to other languages. However, GCNs struggle to model words with long-range dependencies or are not directly connected in the dependency tree. To address these challenges, we propose to utilize the self-attention mechanism where we explicitly fuse structural information to learn the dependencies between words with different syntactic distances. We introduce GATE, a bf Graph bf Attention bf Transformer bf Encoder, and test its cross-lingual transferability on relation and event extraction tasks. We perform experiments on the ACE05 dataset that includes three typologically different languages: English, Chinese, and Arabic. The evaluation results show that GATE outperforms three recently proposed methods by a large margin. Our detailed analysis reveals that due to the reliance on syntactic dependencies, GATE produces robust representations that facilitate transfer across languages.