# DCN+: Mixed Objective and Deep Residual Coattention for Question Answering

arXiv: https://arxiv.org/abs/1711.00106 · HTML: https://mlsanity.com/paper/1711.00106/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 12 of 12)

- [Quark: Controllable Text Generation with Reinforced Unlearning](https://mlsanity.com/paper/2205.13636/) (2022)
- [A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods](https://mlsanity.com/paper/2204.03508/) (2022)
- [NoiseQA: Challenge Set Evaluation for User-Centric Question Answering](https://mlsanity.com/paper/2102.08345/) (2021)
- [Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generation](https://mlsanity.com/paper/2401.07382/) (2024)
- [On the Usability of Transformers-based models for a French Question-Answering task](https://mlsanity.com/paper/2207.09150/) (2022)
- [Adaptive loose optimization for robust question answering](https://mlsanity.com/paper/2305.03971/) (2023)
- [Successor Features for Efficient Multisubject Controlled Text Generation](https://mlsanity.com/paper/2311.04921/) (2023)
- [Polarity Calibration for Opinion Summarization](https://mlsanity.com/paper/2404.01706/) (2024)
- [Bootstrapped Pre-training with Dynamic Identifier Prediction for Generative Retrieval](https://mlsanity.com/paper/2407.11504/) (2024)
- [Multi Document Reading Comprehension](https://mlsanity.com/paper/2201.01706/) (2022)
- [Modular Approach to Machine Reading Comprehension: Mixture of Task-Aware Experts](https://mlsanity.com/paper/2210.01750/) (2022)
- [KILDST: Effective Knowledge-Integrated Learning for Dialogue State Tracking using Gazetteer and Speaker Information](https://mlsanity.com/paper/2301.07341/) (2023)

## Abstract (arXiv)

Traditional models for question answering optimize using cross entropy loss, which encourages exact answers at the cost of penalizing nearby or overlapping answers that are sometimes equally accurate. We propose a mixed objective that combines cross entropy loss with self-critical policy learning. The objective uses rewards derived from word overlap to solve the misalignment between evaluation metric and optimization objective. In addition to the mixed objective, we improve dynamic coattention networks (DCN) with a deep residual coattention encoder that is inspired by recent work in deep self-attention and residual networks. Our proposals improve model performance across question types and input lengths, especially for long questions that requires the ability to capture long-term dependencies. On the Stanford Question Answering Dataset, our model achieves state-of-the-art results with 75.1% exact match accuracy and 83.1% F1, while the ensemble obtains 78.9% exact match accuracy and 86.0% F1.