# Coreferential Reasoning Learning for Language Representation

arXiv: https://arxiv.org/abs/2004.06870 · HTML: https://mlsanity.com/paper/2004.06870/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2020.

## Built on by (most cited first, 30 of 80)

- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Document-level Relation Extraction as Semantic Segmentation](https://mlsanity.com/paper/2106.03618/) (2021)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction](https://mlsanity.com/paper/2102.10249/) (2021)
- [Few-Shot Question Answering by Pretraining Span Selection](https://mlsanity.com/paper/2101.00438/) (2021)
- [DialFact: A Benchmark for Fact-Checking in Dialogue](https://mlsanity.com/paper/2110.08222/) (2021)
- [ProoFVer: Natural Logic Theorem Proving for Fact Verification](https://mlsanity.com/paper/2108.11357/) (2021)
- [CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2401.17043/) (2024)
- [Learning to Retrieve Passages without Supervision](https://mlsanity.com/paper/2112.07708/) (2021)
- [Synthetic Disinformation Attacks on Automated Fact Verification Systems](https://mlsanity.com/paper/2202.09381/) (2022)
- [Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation](https://mlsanity.com/paper/2203.10900/) (2022)
- [An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning](https://mlsanity.com/paper/2102.05980/) (2021)
- [SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2106.01709/) (2021)
- [Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion](https://mlsanity.com/paper/2106.08657/) (2021)
- [Graph Based Network with Contextualized Representations of Turns in Dialogue](https://mlsanity.com/paper/2109.04008/) (2021)
- [SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction](https://mlsanity.com/paper/2109.12093/) (2021)
- [LIQUID: A Framework for List Question Answering Dataset Generation](https://mlsanity.com/paper/2302.01691/) (2023)
- [DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population](https://mlsanity.com/paper/2201.03335/) (2022)
- [Prompt Tuning for Discriminative Pre-trained Language Models](https://mlsanity.com/paper/2205.11166/) (2022)
- [QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions](https://mlsanity.com/paper/2109.12655/) (2021)
- [Document-Level Relation Extraction with Sentences Importance Estimation and Focusing](https://mlsanity.com/paper/2204.12679/) (2022)
- [Learning Logic Rules for Document-level Relation Extraction](https://mlsanity.com/paper/2111.05407/) (2021)
- [A Multi-Level Attention Model for Evidence-Based Fact Checking](https://mlsanity.com/paper/2106.00950/) (2021)
- [Can Pre-trained Language Models Interpret Similes as Smart as Human?](https://mlsanity.com/paper/2203.08452/) (2022)
- [Fact-Saboteurs: A Taxonomy of Evidence Manipulation Attacks against Fact-Verification Systems](https://mlsanity.com/paper/2209.03755/) (2022)
- [GLiREL -- Generalist Model for Zero-Shot Relation Extraction](https://mlsanity.com/paper/2501.03172/) (2025)
- [BEVERS: A General, Simple, and Performant Framework for Automatic Fact Verification](https://mlsanity.com/paper/2303.16974/) (2023)
- [Document-level Relation Extraction with Cross-sentence Reasoning Graph](https://mlsanity.com/paper/2303.03912/) (2023)
- [Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks](https://mlsanity.com/paper/2204.10615/) (2022)
- [None Class Ranking Loss for Document-Level Relation Extraction](https://mlsanity.com/paper/2205.00476/) (2022)

## Abstract (arXiv)

Language representation models such as BERT could effectively capture contextual semantic information from plain text, and have been proved to achieve promising results in lots of downstream NLP tasks with appropriate fine-tuning. However, most existing language representation models cannot explicitly handle coreference, which is essential to the coherent understanding of the whole discourse. To address this issue, we present CorefBERT, a novel language representation model that can capture the coreferential relations in context. The experimental results show that, compared with existing baseline models, CorefBERT can achieve significant improvements consistently on various downstream NLP tasks that require coreferential reasoning, while maintaining comparable performance to previous models on other common NLP tasks. The source code and experiment details of this paper can be obtained from https://github.com/thunlp/CorefBERT.