# Ensemble Transfer Learning for Multilingual Coreference Resolution

arXiv: https://arxiv.org/abs/2301.09175 · HTML: https://mlsanity.com/paper/2301.09175/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 67 references that are in the map.
- First posted on arXiv 22 Jan 2023.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [End-to-end Neural Coreference Resolution](https://mlsanity.com/paper/1707.07045/) (2017)
- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Higher-order Coreference Resolution with Coarse-to-fine Inference](https://mlsanity.com/paper/1804.05392/) (2018)
- [BERT for Coreference Resolution: Baselines and Analysis](https://mlsanity.com/paper/1908.09091/) (2019)
- [Coreference Resolution as Query-based Span Prediction](https://mlsanity.com/paper/1911.01746/) (2019)
- [PyTorch Metric Learning](https://mlsanity.com/paper/2008.09164/) (2020)
- [Revealing the Myth of Higher-Order Inference in Coreference Resolution](https://mlsanity.com/paper/2009.12013/) (2020)
- [Coreference Resolution without Span Representations](https://mlsanity.com/paper/2101.00434/) (2021)
- [Word-Level Coreference Resolution](https://mlsanity.com/paper/2109.04127/) (2021)
- [Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal](https://mlsanity.com/paper/2203.12574/) (2022)
- [Neural Models for Reasoning over Multiple Mentions using Coreference](https://mlsanity.com/paper/1804.05922/) (2018)
- [WEC: Deriving a Large-scale Cross-document Event Coreference dataset from Wikipedia](https://mlsanity.com/paper/2104.05022/) (2021)
- [Interconnected Question Generation with Coreference Alignment and Conversation Flow Modeling](https://mlsanity.com/paper/1906.06893/) (2019)
- [Moving on from OntoNotes: Coreference Resolution Model Transfer](https://mlsanity.com/paper/2104.08457/) (2021)
- [Multilingual Coreference Resolution with Harmonized Annotations](https://mlsanity.com/paper/2107.12088/) (2021)
- [Task-adaptive Pre-training and Self-training are Complementary for Natural Language Understanding](https://mlsanity.com/paper/2109.06466/) (2021)
- [A Simple but Effective BERT Model for Dialog State Tracking on Resource-Limited Systems](https://mlsanity.com/paper/1910.12995/) (2019)
- [Cross-Language Transfer Learning, Continuous Learning, and Domain Adaptation for End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2005.04290/) (2020)

## Abstract (arXiv)

Entity coreference resolution is an important research problem with many applications, including information extraction and question answering. Coreference resolution for English has been studied extensively. However, there is relatively little work for other languages. A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data. To overcome this challenge, we design a simple but effective ensemble-based framework that combines various transfer learning (TL) techniques. We first train several models using different TL methods. Then, during inference, we compute the unweighted average scores of the models' predictions to extract the final set of predicted clusters. Furthermore, we also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts. Leveraging the idea that the coreferential links naturally exist between anchor texts pointing to the same article, our method builds a sizeable distantly-supervised dataset for the target language that consists of tens of thousands of documents. We can pre-train a model on the pseudo-labeled dataset before finetuning it on the final target dataset. Experimental results on two benchmark datasets, OntoNotes and SemEval, confirm the effectiveness of our methods. Our best ensembles consistently outperform the baseline approach of simple training by up to 7.68% in the F1 score. These ensembles also achieve new state-of-the-art results for three languages: Arabic, Dutch, and Spanish.