# Deep Reinforcement Learning for Mention-Ranking Coreference Models

arXiv: https://arxiv.org/abs/1609.08667 · HTML: https://mlsanity.com/paper/1609.08667/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2016.

## Built on by (most cited first, 30 of 63)

- [GenIE: Generative Information Extraction](https://mlsanity.com/paper/2112.08340/) (2021)
- [VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution](https://mlsanity.com/paper/2306.12424/) (2023)
- [CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference](https://mlsanity.com/paper/2406.17626/) (2024)
- [ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Historical News Collections](https://mlsanity.com/paper/2109.03438/) (2021)
- [Automatic Story Generation: Challenges and Attempts](https://mlsanity.com/paper/2102.12634/) (2021)
- [Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2212.04634/) (2022)
- [Moving on from OntoNotes: Coreference Resolution Model Transfer](https://mlsanity.com/paper/2104.08457/) (2021)
- [OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres](https://mlsanity.com/paper/2106.00933/) (2021)
- [Towards Socially Intelligent Agents with Mental State Transition and Human Utility](https://mlsanity.com/paper/2103.07011/) (2021)
- [CREAD: Combined Resolution of Ellipses and Anaphora in Dialogues](https://mlsanity.com/paper/2105.09914/) (2021)
- [Machine Reading, Fast and Slow: When Do Models "Understand" Language?](https://mlsanity.com/paper/2209.07430/) (2022)
- [Sequential Cross-Document Coreference Resolution](https://mlsanity.com/paper/2104.08413/) (2021)
- [To Protect and To Serve? Analyzing Entity-Centric Framing of Police Violence](https://mlsanity.com/paper/2109.05325/) (2021)
- [Discovering Differences in the Representation of People using Contextualized Semantic Axes](https://mlsanity.com/paper/2210.12170/) (2022)
- [AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference](https://mlsanity.com/paper/2302.00762/) (2023)
- [Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues](https://mlsanity.com/paper/2103.00820/) (2021)
- [VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Tasks](https://mlsanity.com/paper/2104.07921/) (2021)
- [A Survey on Deep Reinforcement Learning for Data Processing and Analytics](https://mlsanity.com/paper/2108.04526/) (2021)
- [DEArt: Dataset of European Art](https://mlsanity.com/paper/2211.01226/) (2022)
- [[Lions: 1] and [Tigers: 2] and [Bears: 3], Oh My! Literary Coreference Annotation with LLMs](https://mlsanity.com/paper/2401.17922/) (2024)
- [Reinforcement Learning based Collective Entity Alignment with Adaptive Features](https://mlsanity.com/paper/2101.01353/) (2021)
- [End-to-end Neural Coreference Resolution Revisited: A Simple yet Effective Baseline](https://mlsanity.com/paper/2107.01700/) (2021)
- [The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training](https://mlsanity.com/paper/2205.12502/) (2022)
- [FRAMM: Fair Ranking with Missing Modalities for Clinical Trial Site Selection](https://mlsanity.com/paper/2305.19407/) (2023)
- [Proto: A Neural Cocktail for Generating Appealing Conversations](https://mlsanity.com/paper/2109.02513/) (2021)
- [Better Call the Plumber: Orchestrating Dynamic Information Extraction Pipelines](https://mlsanity.com/paper/2102.10966/) (2021)
- [Can we Fix the Scope for Coreference? Problems and Solutions for Benchmarks beyond OntoNotes](https://mlsanity.com/paper/2112.09742/) (2021)
- [Stylized Knowledge-Grounded Dialogue Generation via Disentangled Template Rewriting](https://mlsanity.com/paper/2204.05610/) (2022)
- [The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources in Natural Language Understanding Systems](https://mlsanity.com/paper/2212.08192/) (2022)
- [A Case Study for Compliance as Code with Graphs and Language Models: Public release of the Regulatory Knowledge Graph](https://mlsanity.com/paper/2302.01842/) (2023)

## Abstract (arXiv)

Coreference resolution systems are typically trained with heuristic loss functions that require careful tuning. In this paper we instead apply reinforcement learning to directly optimize a neural mention-ranking model for coreference evaluation metrics. We experiment with two approaches: the REINFORCE policy gradient algorithm and a reward-rescaled max-margin objective. We find the latter to be more effective, resulting in significant improvements over the current state-of-the-art on the English and Chinese portions of the CoNLL 2012 Shared Task.