# Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation

arXiv: https://arxiv.org/abs/2010.14271 · HTML: https://mlsanity.com/paper/2010.14271/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2020.

## Built on by (most cited first, 7 of 7)

- [From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension](https://mlsanity.com/paper/2112.04735/) (2021)
- [Toward Student-Oriented Teacher Network Training For Knowledge Distillation](https://mlsanity.com/paper/2206.06661/) (2022)
- [Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension](https://mlsanity.com/paper/2204.00996/) (2022)
- [AMTSS: An Adaptive Multi-Teacher Single-Student Knowledge Distillation Framework For Multilingual Language Inference](https://mlsanity.com/paper/2305.07928/) (2023)
- [Promoting Generalized Cross-lingual Question Answering in Few-resource Scenarios via Self-knowledge Distillation](https://mlsanity.com/paper/2309.17134/) (2023)
- [Task Integration Distillation for Object Detectors](https://mlsanity.com/paper/2404.01699/) (2024)
- [Test-Time Code-Switching for Cross-lingual Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2501.14144/) (2025)

## Abstract (arXiv)

Cross-lingual Machine Reading Comprehension (CLMRC) remains a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi, and Vietnamese. Many previous approaches use translation data by translating from a rich-source language, such as English, to low-source languages as auxiliary supervision. However, how to effectively leverage translation data and reduce the impact of noise introduced by translation remains onerous. In this paper, we tackle this challenge and enhance the cross-lingual transferring performance by a novel augmentation approach named Language Branch Machine Reading Comprehension (LBMRC). A language branch is a group of passages in one single language paired with questions in all target languages. We train multiple machine reading comprehension (MRC) models proficient in individual language based on LBMRC. Then, we devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages. Combining the LBMRC and multilingual distillation can be more robust to the data noises, therefore, improving the model's cross-lingual ability. Meanwhile, the produced single multilingual model is applicable to all target languages, which saves the cost of training, inference, and maintenance for multiple models. Extensive experiments on two CLMRC benchmarks clearly show the effectiveness of our proposed method.