# Investigating Recent Large Language Models for Vietnamese Machine Reading Comprehension

arXiv: https://arxiv.org/abs/2503.18062 · HTML: https://mlsanity.com/paper/2503.18062/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 23 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [English Machine Reading Comprehension Datasets: A Survey](https://mlsanity.com/paper/2101.10421/) (2021)
- [Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models](https://mlsanity.com/paper/2006.15994/) (2020)
- [MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension](https://mlsanity.com/paper/1910.00458/) (2019)
- [DUMA: Reading Comprehension with Transposition Thinking](https://mlsanity.com/paper/2001.09415/) (2020)
- [Do Generative Large Language Models need billions of parameters?](https://mlsanity.com/paper/2309.06589/) (2023)
- [Evaluating the Symbol Binding Ability of Large Language Models for Multiple-Choice Questions in Vietnamese General Education](https://mlsanity.com/paper/2310.12059/) (2023)
- [Enhancing lexical-based approach with external knowledge for Vietnamese multiple-choice machine reading comprehension](https://mlsanity.com/paper/2001.05687/) (2020)

## Abstract (arXiv)

Large Language Models (LLMs) have shown remarkable proficiency in Machine Reading Comprehension (MRC) tasks; however, their effectiveness for low-resource languages like Vietnamese remains largely unexplored. In this paper, we fine-tune and evaluate two state-of-the-art LLMs: Llama 3 (8B parameters) and Gemma (7B parameters), on ViMMRC, a Vietnamese MRC dataset. By utilizing Quantized Low-Rank Adaptation (QLoRA), we efficiently fine-tune these models and compare their performance against powerful LLM-based baselines. Although our fine-tuned models are smaller than GPT-3 and GPT-3.5, they outperform both traditional BERT-based approaches and these larger models. This demonstrates the effectiveness of our fine-tuning process, showcasing how modern LLMs can surpass the capabilities of older models like BERT while still being suitable for deployment in resource-constrained environments. Through intensive analyses, we explore various aspects of model performance, providing valuable insights into adapting LLMs for low-resource languages like Vietnamese. Our study contributes to the advancement of natural language processing in low-resource languages, and we make our fine-tuned models publicly available at: https://huggingface.co/iaiuet.