# FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension

arXiv: https://arxiv.org/abs/1711.07341 · HTML: https://mlsanity.com/paper/1711.07341/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2017.

## Built on by (most cited first, 22 of 22)

- [Video Corpus Moment Retrieval with Contrastive Learning](https://mlsanity.com/paper/2105.06247/) (2021)
- [Natural Language Video Localization: A Revisit in Span-based Question Answering Framework](https://mlsanity.com/paper/2102.13558/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [End-to-end Spoken Conversational Question Answering: Task, Dataset and Model](https://mlsanity.com/paper/2204.14272/) (2022)
- [Self-supervised Dialogue Learning for Spoken Conversational Question Answering](https://mlsanity.com/paper/2106.02182/) (2021)
- [NoiseQA: Challenge Set Evaluation for User-Centric Question Answering](https://mlsanity.com/paper/2102.08345/) (2021)
- [Parallel Attention Network with Sequence Matching for Video Grounding](https://mlsanity.com/paper/2105.08481/) (2021)
- [Do Question Answering Modeling Improvements Hold Across Benchmarks?](https://mlsanity.com/paper/2102.01065/) (2021)
- [Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos](https://mlsanity.com/paper/2208.01954/) (2022)
- [On the Usability of Transformers-based models for a French Question-Answering task](https://mlsanity.com/paper/2207.09150/) (2022)
- [Coarse-grained decomposition and fine-grained interaction for multi-hop question answering](https://mlsanity.com/paper/2101.05988/) (2021)
- [ComQA:Compositional Question Answering via Hierarchical Graph Neural Networks](https://mlsanity.com/paper/2101.06400/) (2021)
- [Data-Centric Improvements for Enhancing Multi-Modal Understanding in Spoken Conversation Modeling](https://mlsanity.com/paper/2412.15995/) (2024)
- [Interactive Machine Comprehension with Dynamic Knowledge Graphs](https://mlsanity.com/paper/2109.00077/) (2021)
- [Momentum Contrastive Pre-training for Question Answering](https://mlsanity.com/paper/2212.05762/) (2022)
- [Lite Unified Modeling for Discriminative Reading Comprehension](https://mlsanity.com/paper/2203.14103/) (2022)
- [Tracing Origins: Coreference-aware Machine Reading Comprehension](https://mlsanity.com/paper/2110.07961/) (2021)
- [An Automated Question-Answering Framework Based on Evolution Algorithm](https://mlsanity.com/paper/2201.10797/) (2022)
- [Improving Lexical Embeddings for Robust Question Answering](https://mlsanity.com/paper/2202.13636/) (2022)
- [Exploring and Exploiting Multi-Granularity Representations for Machine Reading Comprehension](https://mlsanity.com/paper/2208.08750/) (2022)
- [AGent: A Novel Pipeline for Automatically Creating Unanswerable Questions](https://mlsanity.com/paper/2309.05103/) (2023)

## Abstract (arXiv)

This paper introduces a new neural structure called FusionNet, which extends existing attention approaches from three perspectives. First, it puts forward a novel concept of "history of word" to characterize attention information from the lowest word-level embedding up to the highest semantic-level representation. Second, it introduces an improved attention scoring function that better utilizes the "history of word" concept. Third, it proposes a fully-aware multi-level attention mechanism to capture the complete information in one text (such as a question) and exploit it in its counterpart (such as context or passage) layer by layer. We apply FusionNet to the Stanford Question Answering Dataset (SQuAD) and it achieves the first position for both single and ensemble model on the official SQuAD leaderboard at the time of writing (Oct. 4th, 2017). Meanwhile, we verify the generalization of FusionNet with two adversarial SQuAD datasets and it sets up the new state-of-the-art on both datasets: on AddSent, FusionNet increases the best F1 metric from 46.6% to 51.4%; on AddOneSent, FusionNet boosts the best F1 metric from 56.0% to 60.7%.