# Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehension

arXiv: https://arxiv.org/abs/2012.10877 · HTML: https://mlsanity.com/paper/2012.10877/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2020.

## Built on by (most cited first, 22 of 22)

- [Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2105.07059/) (2021)
- [Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation](https://mlsanity.com/paper/2111.04318/) (2021)
- [Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering](https://mlsanity.com/paper/2109.03381/) (2021)
- [End-to-end Spoken Conversational Question Answering: Task, Dataset and Model](https://mlsanity.com/paper/2204.14272/) (2022)
- [Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations](https://mlsanity.com/paper/2402.11975/) (2024)
- [Self-supervised Dialogue Learning for Spoken Conversational Question Answering](https://mlsanity.com/paper/2106.02182/) (2021)
- [From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension](https://mlsanity.com/paper/2112.04735/) (2021)
- [A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles](https://mlsanity.com/paper/2111.00278/) (2021)
- [EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System](https://mlsanity.com/paper/2206.13441/) (2022)
- [FiTs: Fine-grained Two-stage Training for Knowledge-aware Question Answering](https://mlsanity.com/paper/2302.11799/) (2023)
- [Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling](https://mlsanity.com/paper/2204.05210/) (2022)
- [Graph-in-Graph Network for Automatic Gene Ontology Description Generation](https://mlsanity.com/paper/2206.05311/) (2022)
- [The End of Reward Engineering: How LLMs Are Redefining Multi-Agent Coordination](https://mlsanity.com/paper/2601.08237/) (2026)
- [Adaptive loose optimization for robust question answering](https://mlsanity.com/paper/2305.03971/) (2023)
- [NeuroSonic: Conditional Flow Matching for EEG-to-Speech Reconstruction](https://mlsanity.com/paper/2606.24087/) (2026)
- [Lite Unified Modeling for Discriminative Reading Comprehension](https://mlsanity.com/paper/2203.14103/) (2022)
- [Low-resource Accent Classification in Geographically-proximate Settings: A Forensic and Sociophonetics Perspective](https://mlsanity.com/paper/2206.12759/) (2022)
- [Natural Response Generation for Chinese Reading Comprehension](https://mlsanity.com/paper/2302.08817/) (2023)
- [Orca: A Few-shot Benchmark for Chinese Conversational Machine Reading Comprehension](https://mlsanity.com/paper/2302.13619/) (2023)
- [EMVLight: A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles](https://mlsanity.com/paper/2109.05429/) (2021)
- [Exploring and Exploiting Multi-Granularity Representations for Machine Reading Comprehension](https://mlsanity.com/paper/2208.08750/) (2022)
- [Feature-augmented Machine Reading Comprehension with Auxiliary Tasks](https://mlsanity.com/paper/2211.09438/) (2022)

## Abstract (arXiv)

Recently, the attention-enhanced multi-layer encoder, such as Transformer, has been extensively studied in Machine Reading Comprehension (MRC). To predict the answer, it is common practice to employ a predictor to draw information only from the final encoder layer which generates the textitcoarse-grained representations of the source sequences, i.e., passage and question. Previous studies have shown that the representation of source sequence becomes more textitcoarse-grained from textitfine-grained as the encoding layer increases. It is generally believed that with the growing number of layers in deep neural networks, the encoding process will gather relevant information for each location increasingly, resulting in more textitcoarse-grained representations, which adds the likelihood of similarity to other locations (referring to homogeneity). Such a phenomenon will mislead the model to make wrong judgments so as to degrade the performance. To this end, we propose a novel approach called Adaptive Bidirectional Attention, which adaptively exploits the source representations of different levels to the predictor. Experimental results on the benchmark dataset, SQuAD 2.0 demonstrate the effectiveness of our approach, and the results are better than the previous state-of-the-art model by 2.5% EM and 2.3% F1 scores.