# Densely Connected Attention Propagation for Reading Comprehension

arXiv: https://arxiv.org/abs/1811.04210 · HTML: https://mlsanity.com/paper/1811.04210/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2018.

## Built on by (most cited first, 7 of 7)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [OmniNet: Omnidirectional Representations from Transformers](https://mlsanity.com/paper/2103.01075/) (2021)
- [Rethinking Search: Making Domain Experts out of Dilettantes](https://mlsanity.com/paper/2105.02274/) (2021)
- [PQuAD: A Persian Question Answering Dataset](https://mlsanity.com/paper/2202.06219/) (2022)
- [Evaluating ChatGPT as a Question Answering System: A Comprehensive Analysis and Comparison with Existing Models](https://mlsanity.com/paper/2312.07592/) (2023)
- [A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering](https://mlsanity.com/paper/2106.07174/) (2021)
- [Sparse-softmax: A Simpler and Faster Alternative Softmax Transformation](https://mlsanity.com/paper/2112.12433/) (2021)

## Abstract (arXiv)

We propose DecaProp (Densely Connected Attention Propagation), a new densely connected neural architecture for reading comprehension (RC). There are two distinct characteristics of our model. Firstly, our model densely connects all pairwise layers of the network, modeling relationships between passage and query across all hierarchical levels. Secondly, the dense connectors in our network are learned via attention instead of standard residual skip-connectors. To this end, we propose novel Bidirectional Attention Connectors (BAC) for efficiently forging connections throughout the network. We conduct extensive experiments on four challenging RC benchmarks. Our proposed approach achieves state-of-the-art results on all four, outperforming existing baselines by up to 2.6%-14.2% in absolute F1 score.