# Multilingual Entity and Relation Extraction from Unified to Language-specific Training

arXiv: https://arxiv.org/abs/2301.04434 · HTML: https://mlsanity.com/paper/2301.04434/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 67 references that are in the map.
- First posted on arXiv 11 Jan 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation](https://mlsanity.com/paper/2004.11867/) (2020)
- [Entity, Relation, and Event Extraction with Contextualized Span Representations](https://mlsanity.com/paper/1909.03546/) (2019)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Span-based Joint Entity and Relation Extraction with Transformer Pre-training](https://mlsanity.com/paper/1909.07755/) (2019)
- [Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism](https://mlsanity.com/paper/1601.01073/) (2016)
- [Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme](https://mlsanity.com/paper/1706.05075/) (2017)
- [Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction](https://mlsanity.com/paper/1911.09886/) (2019)
- [Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information](https://mlsanity.com/paper/2010.03142/) (2020)
- [Neural Open Information Extraction](https://mlsanity.com/paper/1805.04270/) (2018)
- [A Sequence-to-Set Network for Nested Named Entity Recognition](https://mlsanity.com/paper/2105.08901/) (2021)
- [IMoJIE: Iterative Memory-Based Joint Open Information Extraction](https://mlsanity.com/paper/2005.08178/) (2020)
- [Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task](https://mlsanity.com/paper/2111.02086/) (2021)
- [Span Model for Open Information Extraction on Accurate Corpus](https://mlsanity.com/paper/1901.10879/) (2019)
- [LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation](https://mlsanity.com/paper/2210.15461/) (2022)
- [Multilingual NMT with a language-independent attention bridge](https://mlsanity.com/paper/1811.00498/) (2018)
- [GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator](https://mlsanity.com/paper/2212.10218/) (2022)
- [Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification](https://mlsanity.com/paper/2010.16068/) (2020)
- [The RELX Dataset and Matching the Multilingual Blanks for Cross-Lingual Relation Classification](https://mlsanity.com/paper/2010.09381/) (2020)
- [Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling](https://mlsanity.com/paper/1506.07650/) (2015)
- [UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation](https://mlsanity.com/paper/2207.04900/) (2022)

## Built on by (most cited first, 3 of 3)

- [M2C: Towards Automatic Multimodal Manga Complement](https://mlsanity.com/paper/2310.17130/) (2023)
- [Leveraging Large Language Models for Enhanced NLP Task Performance through Knowledge Distillation and Optimized Training Strategies](https://mlsanity.com/paper/2402.09282/) (2024)
- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)

## Abstract (arXiv)

Entity and relation extraction is a key task in information extraction, where the output can be used for downstream NLP tasks. Existing approaches for entity and relation extraction tasks mainly focus on the English corpora and ignore other languages. Thus, it is critical to improving performance in a multilingual setting. Meanwhile, multilingual training is usually used to boost cross-lingual performance by transferring knowledge from languages (e.g., high-resource) to other (e.g., low-resource) languages. However, language interference usually exists in multilingual tasks as the model parameters are shared among all languages. In this paper, we propose a two-stage multilingual training method and a joint model called Multilingual Entity and Relation Extraction framework (mERE) to mitigate language interference across languages. Specifically, we randomly concatenate sentences in different languages to train a Language-universal Aggregator (LA), which narrows the distance of embedding representations by obtaining the unified language representation. Then, we separate parameters to mitigate interference via tuning a Language-specific Switcher (LS), which includes several independent sub-modules to refine the language-specific feature representation. After that, to enhance the relational triple extraction, the sentence representations concatenated with the relation feature are used to recognize the entities. Extensive experimental results show that our method outperforms both the monolingual and multilingual baseline methods. Besides, we also perform detailed analysis to show that mERE is lightweight but effective on relational triple extraction and mERE{} is easy to transfer to other backbone models of multi-field tasks, which further demonstrates the effectiveness of our method.