# Improving Span Representation for Domain-adapted Coreference Resolution

arXiv: https://arxiv.org/abs/2109.09811 · HTML: https://mlsanity.com/paper/2109.09811/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [End-to-end Neural Coreference Resolution](https://mlsanity.com/paper/1707.07045/) (2017)
- [BERT for Coreference Resolution: Baselines and Analysis](https://mlsanity.com/paper/1908.09091/) (2019)
- [E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT](https://mlsanity.com/paper/1911.03681/) (2019)
- [Learning from Context or Names? An Empirical Study on Neural Relation Extraction](https://mlsanity.com/paper/2010.01923/) (2020)
- [Revealing the Myth of Higher-Order Inference in Coreference Resolution](https://mlsanity.com/paper/2009.12013/) (2020)
- [Coreference Resolution without Span Representations](https://mlsanity.com/paper/2101.00434/) (2021)
- [Syntactic Scaffolds for Semantic Structures](https://mlsanity.com/paper/1808.10485/) (2018)
- [Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QA](https://mlsanity.com/paper/2004.03354/) (2020)
- [Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data](https://mlsanity.com/paper/2010.03656/) (2020)
- [Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction](https://mlsanity.com/paper/2004.08134/) (2020)
- [Moving on from OntoNotes: Coreference Resolution Model Transfer](https://mlsanity.com/paper/2104.08457/) (2021)

## Abstract (arXiv)

Recent work has shown fine-tuning neural coreference models can produce strong performance when adapting to different domains. However, at the same time, this can require a large amount of annotated target examples. In this work, we focus on supervised domain adaptation for clinical notes, proposing the use of concept knowledge to more efficiently adapt coreference models to a new domain. We develop methods to improve the span representations via (1) a retrofitting loss to incentivize span representations to satisfy a knowledge-based distance function and (2) a scaffolding loss to guide the recovery of knowledge from the span representation. By integrating these losses, our model is able to improve our baseline precision and F-1 score. In particular, we show that incorporating knowledge with end-to-end coreference models results in better performance on the most challenging, domain-specific spans.