# Integrating Semantic Information into Sketchy Reading Module of Retro-Reader for Vietnamese Machine Reading Comprehension

arXiv: https://arxiv.org/abs/2301.00429 · HTML: https://mlsanity.com/paper/2301.00429/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 1 Jan 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Semantics-aware BERT for Language Understanding](https://mlsanity.com/paper/1909.02209/) (2019)
- [Retrospective Reader for Machine Reading Comprehension](https://mlsanity.com/paper/2001.09694/) (2020)
- [Read + Verify: Machine Reading Comprehension with Unanswerable Questions](https://mlsanity.com/paper/1808.05759/) (2018)
- [A Vietnamese Dataset for Evaluating Machine Reading Comprehension](https://mlsanity.com/paper/2009.14725/) (2020)
- [Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering](https://mlsanity.com/paper/1810.00494/) (2018)
- [New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles](https://mlsanity.com/paper/2006.11138/) (2020)
- [Dependency or Span, End-to-End Uniform Semantic Role Labeling](https://mlsanity.com/paper/1901.05280/) (2019)
- [Text Understanding with the Attention Sum Reader Network](https://mlsanity.com/paper/1603.01547/) (2016)
- [VLSP 2021 - ViMRC Challenge: Vietnamese Machine Reading Comprehension](https://mlsanity.com/paper/2203.11400/) (2022)
- [Stochastic Answer Networks for SQuAD 2.0](https://mlsanity.com/paper/1809.09194/) (2018)
- [Vietnamese Semantic Role Labelling](https://mlsanity.com/paper/1711.10124/) (2017)
- [Building a Semantic Role Labelling System for Vietnamese](https://mlsanity.com/paper/1705.04038/) (2017)

## Abstract (arXiv)

Machine Reading Comprehension has become one of the most advanced and popular research topics in the fields of Natural Language Processing in recent years. The classification of answerability questions is a relatively significant sub-task in machine reading comprehension; however, there haven't been many studies. Retro-Reader is one of the studies that has solved this problem effectively. However, the encoders of most traditional machine reading comprehension models in general and Retro-Reader, in particular, have not been able to exploit the contextual semantic information of the context completely. Inspired by SemBERT, we use semantic role labels from the SRL task to add semantics to pre-trained language models such as mBERT, XLM-R, PhoBERT. This experiment was conducted to compare the influence of semantics on the classification of answerability for the Vietnamese machine reading comprehension. Additionally, we hope this experiment will enhance the encoder for the Retro-Reader model's Sketchy Reading Module. The improved Retro-Reader model's encoder with semantics was first applied to the Vietnamese Machine Reading Comprehension task and obtained positive results.