# WikiCREM: A Large Unsupervised Corpus for Coreference Resolution

arXiv: https://arxiv.org/abs/1908.08025 · HTML: https://mlsanity.com/paper/1908.08025/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2019.

## Built on by (most cited first, 11 of 11)

- [Few-Shot Question Answering by Pretraining Span Selection](https://mlsanity.com/paper/2101.00438/) (2021)
- [Learning to Retrieve Passages without Supervision](https://mlsanity.com/paper/2112.07708/) (2021)
- [Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting](https://mlsanity.com/paper/2110.05367/) (2021)
- [The Defeat of the Winograd Schema Challenge](https://mlsanity.com/paper/2201.02387/) (2022)
- [Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction](https://mlsanity.com/paper/2105.12392/) (2021)
- [Enhancing Natural Language Representation with Large-Scale Out-of-Domain Commonsense](https://mlsanity.com/paper/2109.02572/) (2021)
- [Attention-based Contrastive Learning for Winograd Schemas](https://mlsanity.com/paper/2109.05108/) (2021)
- [Open Rule Induction](https://mlsanity.com/paper/2110.13577/) (2021)
- [Solving the Challenge Set without Solving the Task: On Winograd Schemas as a Test of Pronominal Coreference Resolution](https://mlsanity.com/paper/2410.09448/) (2024)
- [Birdie: Advancing State Space Models with Reward-Driven Objectives and Curricula](https://mlsanity.com/paper/2411.01030/) (2024)
- [Experience and Prediction: A Metric of Hardness for a Novel Litmus Test](https://mlsanity.com/paper/2309.02534/) (2023)

## Abstract (arXiv)

Pronoun resolution is a major area of natural language understanding. However, large-scale training sets are still scarce, since manually labelling data is costly. In this work, we introduce WikiCREM (Wikipedia CoREferences Masked) a large-scale, yet accurate dataset of pronoun disambiguation instances. We use a language-model-based approach for pronoun resolution in combination with our WikiCREM dataset. We compare a series of models on a collection of diverse and challenging coreference resolution problems, where we match or outperform previous state-of-the-art approaches on 6 out of 7 datasets, such as GAP, DPR, WNLI, PDP, WinoBias, and WinoGender. We release our model to be used off-the-shelf for solving pronoun disambiguation.