# Anatomy of OntoGUM--Adapting GUM to the OntoNotes Scheme to Evaluate Robustness of SOTA Coreference Algorithms

arXiv: https://arxiv.org/abs/2110.05727 · HTML: https://mlsanity.com/paper/2110.05727/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 12 Oct 2021.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [End-to-end Neural Coreference Resolution](https://mlsanity.com/paper/1707.07045/) (2017)
- [Higher-order Coreference Resolution with Coarse-to-fine Inference](https://mlsanity.com/paper/1804.05392/) (2018)
- [BERT for Coreference Resolution: Baselines and Analysis](https://mlsanity.com/paper/1908.09091/) (2019)
- [Deep Reinforcement Learning for Mention-Ranking Coreference Models](https://mlsanity.com/paper/1609.08667/) (2016)
- [Improving Coreference Resolution by Learning Entity-Level Distributed Representations](https://mlsanity.com/paper/1606.01323/) (2016)
- [PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution](https://mlsanity.com/paper/1810.09807/) (2018)
- [Learning Global Features for Coreference Resolution](https://mlsanity.com/paper/1604.03035/) (2016)
- [OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres](https://mlsanity.com/paper/2106.00933/) (2021)
- [Lexical Features in Coreference Resolution: To be Used With Caution](https://mlsanity.com/paper/1704.06779/) (2017)

## Built on by (most cited first, 3 of 3)

- [Mention Annotations Alone Enable Efficient Domain Adaptation for Coreference Resolution](https://mlsanity.com/paper/2210.07602/) (2022)
- [Can we Fix the Scope for Coreference? Problems and Solutions for Benchmarks beyond OntoNotes](https://mlsanity.com/paper/2112.09742/) (2021)
- [A Controlled Reevaluation of Coreference Resolution Models](https://mlsanity.com/paper/2404.00727/) (2024)

## Abstract (arXiv)

SOTA coreference resolution produces increasingly impressive scores on the OntoNotes benchmark. However lack of comparable data following the same scheme for more genres makes it difficult to evaluate generalizability to open domain data. Zhu et al. (2021) introduced the creation of the OntoGUM corpus for evaluating geralizability of the latest neural LM-based end-to-end systems. This paper covers details of the mapping process which is a set of deterministic rules applied to the rich syntactic and discourse annotations manually annotated in the GUM corpus. Out-of-domain evaluation across 12 genres shows nearly 15-20% degradation for both deterministic and deep learning systems, indicating a lack of generalizability or covert overfitting in existing coreference resolution models.