# How Good is the Model in Model-in-the-loop Event Coreference Resolution Annotation?

arXiv: https://arxiv.org/abs/2306.05434 · HTML: https://mlsanity.com/paper/2306.05434/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Simple BERT Models for Relation Extraction and Semantic Role Labeling](https://mlsanity.com/paper/1904.05255/) (2019)
- [Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution](https://mlsanity.com/paper/1906.01753/) (2019)
- [Streamlining Cross-Document Coreference Resolution: Evaluation and Modeling](https://mlsanity.com/paper/2009.11032/) (2020)
- [Paraphrasing vs Coreferring: Two Sides of the Same Coin](https://mlsanity.com/paper/2004.14979/) (2020)
- [CoRefi: A Crowd Sourcing Suite for Coreference Annotation](https://mlsanity.com/paper/2010.02588/) (2020)
- [Adapting Coreference Resolution Models through Active Learning](https://mlsanity.com/paper/2104.07611/) (2021)

## Built on by (most cited first, 4 of 4)

- [Linear Cross-document Event Coreference Resolution with X-AMR](https://mlsanity.com/paper/2404.08656/) (2024)
- [Generating Harder Cross-document Event Coreference Resolution Datasets using Metaphoric Paraphrasing](https://mlsanity.com/paper/2407.11988/) (2024)
- [CAMRA: Copilot for AMR Annotation](https://mlsanity.com/paper/2311.10928/) (2023)
- [X-AMR Annotation Tool](https://mlsanity.com/paper/2403.15407/) (2024)

## Abstract (arXiv)

Annotating cross-document event coreference links is a time-consuming and cognitively demanding task that can compromise annotation quality and efficiency. To address this, we propose a model-in-the-loop annotation approach for event coreference resolution, where a machine learning model suggests likely corefering event pairs only. We evaluate the effectiveness of this approach by first simulating the annotation process and then, using a novel annotator-centric Recall-Annotation effort trade-off metric, we compare the results of various underlying models and datasets. We finally present a method for obtaining 97% recall while substantially reducing the workload required by a fully manual annotation process. Code and data can be found at https://github.com/ahmeshaf/modelincoref