# Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-sentence Dependency Graph

arXiv: https://arxiv.org/abs/2112.00503 · HTML: https://mlsanity.com/paper/2112.00503/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [MLQA: Evaluating Cross-lingual Extractive Question Answering](https://mlsanity.com/paper/1910.07475/) (2019)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Graph Transformer for Graph-to-Sequence Learning](https://mlsanity.com/paper/1911.07470/) (2019)
- [Linguistically-Informed Self-Attention for Semantic Role Labeling](https://mlsanity.com/paper/1804.08199/) (2018)
- [Zero-Shot Cross-Lingual Transfer with Meta Learning](https://mlsanity.com/paper/2003.02739/) (2020)
- [A Graph-to-Sequence Model for AMR-to-Text Generation](https://mlsanity.com/paper/1805.02473/) (2018)
- [SG-Net: Syntax-Guided Machine Reading Comprehension](https://mlsanity.com/paper/1908.05147/) (2019)
- [Modeling Graph Structure in Transformer for Better AMR-to-Text Generation](https://mlsanity.com/paper/1909.00136/) (2019)
- [Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning](https://mlsanity.com/paper/1908.05957/) (2019)
- [Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model](https://mlsanity.com/paper/1909.09587/) (2019)
- [Cross-Lingual Machine Reading Comprehension](https://mlsanity.com/paper/1909.00361/) (2019)
- [Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension](https://mlsanity.com/paper/2004.14069/) (2020)
- [Syntax-Directed Attention for Neural Machine Translation](https://mlsanity.com/paper/1711.04231/) (2017)
- [Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations](https://mlsanity.com/paper/1905.02878/) (2019)
- [Syntax-aware Neural Semantic Role Labeling](https://mlsanity.com/paper/1907.09312/) (2019)
- [Syntax-aware Neural Semantic Role Labeling with Supertags](https://mlsanity.com/paper/1903.05260/) (2019)
- [Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty Estimation](https://mlsanity.com/paper/2109.00194/) (2021)

## Built on by (most cited first, 2 of 2)

- [Fine-Grained Modeling of Narrative Context: A Coherence Perspective via Retrospective Questions](https://mlsanity.com/paper/2402.13551/) (2024)
- [Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer](https://mlsanity.com/paper/2404.16627/) (2024)

## Abstract (arXiv)

We target the task of cross-lingual Machine Reading Comprehension (MRC) in the direct zero-shot setting, by incorporating syntactic features from Universal Dependencies (UD), and the key features we use are the syntactic relations within each sentence. While previous work has demonstrated effective syntax-guided MRC models, we propose to adopt the inter-sentence syntactic relations, in addition to the rudimentary intra-sentence relations, to further utilize the syntactic dependencies in the multi-sentence input of the MRC task. In our approach, we build the Inter-Sentence Dependency Graph (ISDG) connecting dependency trees to form global syntactic relations across sentences. We then propose the ISDG encoder that encodes the global dependency graph, addressing the inter-sentence relations via both one-hop and multi-hop dependency paths explicitly. Experiments on three multilingual MRC datasets (XQuAD, MLQA, TyDiQA-GoldP) show that our encoder that is only trained on English is able to improve the zero-shot performance on all 14 test sets covering 8 languages, with up to 3.8 F1 / 5.2 EM improvement on-average, and 5.2 F1 / 11.2 EM on certain languages. Further analysis shows the improvement can be attributed to the attention on the cross-linguistically consistent syntactic path.