# Aggregating Crowdsourced and Automatic Judgments to Scale Up a Corpus of Anaphoric Reference for Fiction and Wikipedia Texts

arXiv: https://arxiv.org/abs/2210.05581 · HTML: https://mlsanity.com/paper/2210.05581/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 69 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [End-to-end Neural Coreference Resolution](https://mlsanity.com/paper/1707.07045/) (2017)
- [Higher-order Coreference Resolution with Coarse-to-fine Inference](https://mlsanity.com/paper/1804.05392/) (2018)
- [An Annotated Dataset of Coreference in English Literature](https://mlsanity.com/paper/1912.01140/) (2019)
- [PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution](https://mlsanity.com/paper/1810.09807/) (2018)
- [Moving on from OntoNotes: Coreference Resolution Model Transfer](https://mlsanity.com/paper/2104.08457/) (2021)
- [A Cluster Ranking Model for Full Anaphora Resolution](https://mlsanity.com/paper/1911.09532/) (2019)
- [Active Learning for Coreference Resolution using Discrete Annotation](https://mlsanity.com/paper/2004.13671/) (2020)
- [Adapting Coreference Resolution Models through Active Learning](https://mlsanity.com/paper/2104.07611/) (2021)
- [AMALGUM -- A Free, Balanced, Multilayer English Web Corpus](https://mlsanity.com/paper/2006.10677/) (2020)
- [Data Augmentation Methods for Anaphoric Zero Pronouns](https://mlsanity.com/paper/2109.09825/) (2021)

## Built on by (most cited first, 3 of 3)

- [Large Language Models Fall Short: Understanding Complex Relationships in Detective Narratives](https://mlsanity.com/paper/2402.11051/) (2024)
- [Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspective](https://mlsanity.com/paper/2303.09092/) (2023)
- [ezCoref: Towards Unifying Annotation Guidelines for Coreference Resolution](https://mlsanity.com/paper/2210.07188/) (2022)

## Abstract (arXiv)

Although several datasets annotated for anaphoric reference/coreference exist, even the largest such datasets have limitations in terms of size, range of domains, coverage of anaphoric phenomena, and size of documents included. Yet, the approaches proposed to scale up anaphoric annotation haven't so far resulted in datasets overcoming these limitations. In this paper, we introduce a new release of a corpus for anaphoric reference labelled via a game-with-a-purpose. This new release is comparable in size to the largest existing corpora for anaphoric reference due in part to substantial activity by the players, in part thanks to the use of a new resolve-and-aggregate paradigm to 'complete' markable annotations through the combination of an anaphoric resolver and an aggregation method for anaphoric reference. The proposed method could be adopted to greatly speed up annotation time in other projects involving games-with-a-purpose. In addition, the corpus covers genres for which no comparable size datasets exist (Fiction and Wikipedia); it covers singletons and non-referring expressions; and it includes a substantial number of long documents (> 2K in length).