# SG-Net: Syntax-Guided Machine Reading Comprehension

arXiv: https://arxiv.org/abs/1908.05147 · HTML: https://mlsanity.com/paper/1908.05147/

## Key facts

- Cited by 52 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 14 Aug 2019.

## Built on by (most cited first, 30 of 52)

- [Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale](https://mlsanity.com/paper/2203.00633/) (2022)
- [End-to-end Spoken Conversational Question Answering: Task, Dataset and Model](https://mlsanity.com/paper/2204.14272/) (2022)
- [Capturing Row and Column Semantics in Transformer Based Question Answering over Tables](https://mlsanity.com/paper/2104.08303/) (2021)
- [Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrifice](https://mlsanity.com/paper/2105.14553/) (2021)
- [Discriminative Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2106.01562/) (2021)
- [DICE: Diverse Diffusion Model with Scoring for Trajectory Prediction](https://mlsanity.com/paper/2310.14570/) (2023)
- [CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks](https://mlsanity.com/paper/2310.14623/) (2023)
- [Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization](https://mlsanity.com/paper/2305.16784/) (2023)
- [Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns](https://mlsanity.com/paper/2310.01749/) (2023)
- [Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2201.04831/) (2022)
- [Syntax-augmented Multilingual BERT for Cross-lingual Transfer](https://mlsanity.com/paper/2106.02134/) (2021)
- [Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge](https://mlsanity.com/paper/2102.05474/) (2021)
- [Multimodal Graph Transformer for Multimodal Question Answering](https://mlsanity.com/paper/2305.00581/) (2023)
- [Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.00731/) (2022)
- [Adversarial Self-Attention for Language Understanding](https://mlsanity.com/paper/2206.12608/) (2022)
- [Text Alignment Is An Efficient Unified Model for Massive NLP Tasks](https://mlsanity.com/paper/2307.02729/) (2023)
- [Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models](https://mlsanity.com/paper/2407.17406/) (2024)
- [Not All Attention Is All You Need](https://mlsanity.com/paper/2104.04692/) (2021)
- [Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking](https://mlsanity.com/paper/2107.12578/) (2021)
- [Fact-driven Logical Reasoning for Machine Reading Comprehension](https://mlsanity.com/paper/2105.10334/) (2021)
- [A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction](https://mlsanity.com/paper/2111.03294/) (2021)
- [DTW at Qur'an QA 2022: Utilising Transfer Learning with Transformers for Question Answering in a Low-resource Domain](https://mlsanity.com/paper/2205.06025/) (2022)
- [Multi-turn Dialogue Comprehension from a Topic-aware Perspective](https://mlsanity.com/paper/2309.09666/) (2023)
- [Adapting Pre-trained Generative Models for Extractive Question Answering](https://mlsanity.com/paper/2311.02961/) (2023)
- [Towards Confident Machine Reading Comprehension](https://mlsanity.com/paper/2101.07942/) (2021)
- [When Retriever-Reader Meets Scenario-Based Multiple-Choice Questions](https://mlsanity.com/paper/2108.13875/) (2021)
- [FedQAS: Privacy-aware machine reading comprehension with federated learning](https://mlsanity.com/paper/2202.04742/) (2022)
- [Targeted Extraction of Temporal Facts from Textual Resources for Improved Temporal Question Answering over Knowledge Bases](https://mlsanity.com/paper/2203.11054/) (2022)
- [Syntax-guided Localized Self-attention by Constituency Syntactic Distance](https://mlsanity.com/paper/2210.11759/) (2022)
- [MCR-Net: A Multi-Step Co-Interactive Relation Network for Unanswerable Questions on Machine Reading Comprehension](https://mlsanity.com/paper/2103.04567/) (2021)

## Abstract (arXiv)

For machine reading comprehension, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy passages and getting ride of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanism for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. To verify its effectiveness, the proposed SG-Net is applied to typical pre-trained language model BERT which is right based on a Transformer encoder. Extensive experiments on popular benchmarks including SQuAD 2.0 and RACE show that the proposed SG-Net design helps achieve substantial performance improvement over strong baselines.