# A Benchmark Dataset and Evaluation Framework for Vietnamese Large Language Models in Customer Support

arXiv: https://arxiv.org/abs/2507.22542 · HTML: https://mlsanity.com/paper/2507.22542/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 30 Jul 2025.

## Builds on (most cited first, 19 of 19)

- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Efficient Large Language Models: A Survey](https://mlsanity.com/paper/2312.03863/) (2023)
- [Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier](https://mlsanity.com/paper/2412.04261/) (2024)
- [SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languages](https://mlsanity.com/paper/2407.19672/) (2024)
- [A Vietnamese Dataset for Evaluating Machine Reading Comprehension](https://mlsanity.com/paper/2009.14725/) (2020)
- [New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles](https://mlsanity.com/paper/2006.11138/) (2020)
- [Sailor: Open Language Models for South-East Asia](https://mlsanity.com/paper/2404.03608/) (2024)
- [VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models](https://mlsanity.com/paper/2305.12199/) (2023)
- [VinaLLaMA: LLaMA-based Vietnamese Foundation Model](https://mlsanity.com/paper/2312.11011/) (2023)
- [Sentence Extraction-Based Machine Reading Comprehension for Vietnamese](https://mlsanity.com/paper/2105.09043/) (2021)
- [Conversational Machine Reading Comprehension for Vietnamese Healthcare Texts](https://mlsanity.com/paper/2105.01542/) (2021)
- [Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models](https://mlsanity.com/paper/2403.02715/) (2024)
- [SPBERTQA: A Two-Stage Question Answering System Based on Sentence Transformers for Medical Texts](https://mlsanity.com/paper/2206.09600/) (2022)
- [UIT-ViCoV19QA: A Dataset for COVID-19 Community-based Question Answering on Vietnamese Language](https://mlsanity.com/paper/2209.06668/) (2022)
- [Using Large Language Models for education managements in Vietnamese with low resources](https://mlsanity.com/paper/2501.15022/) (2025)
- [Improving Vietnamese Legal Question--Answering System based on Automatic Data Enrichment](https://mlsanity.com/paper/2306.04841/) (2023)
- [R2GQA: Retriever-Reader-Generator Question Answering System to Support Students Understanding Legal Regulations in Higher Education](https://mlsanity.com/paper/2409.02840/) (2024)
- [VlogQA: Task, Dataset, and Baseline Models for Vietnamese Spoken-Based Machine Reading Comprehension](https://mlsanity.com/paper/2402.02655/) (2024)

## Abstract (arXiv)

With the rapid growth of Artificial Intelligence, Large Language Models (LLMs) have become essential for Question Answering (QA) systems, improving efficiency and reducing human workload in customer service. The emergence of Vietnamese LLMs (ViLLMs) highlights lightweight open-source models as a practical choice for their accuracy, efficiency, and privacy benefits. However, domain-specific evaluations remain limited, and the absence of benchmark datasets reflecting real customer interactions makes it difficult for enterprises to select suitable models for support applications. To address this gap, we introduce the Customer Support Conversations Dataset (CSConDa), a curated benchmark of over 9,000 QA pairs drawn from real interactions with human advisors at a large Vietnamese software company. Covering diverse topics such as pricing, product availability, and technical troubleshooting, CSConDa provides a representative basis for evaluating ViLLMs in practical scenarios. We further present a comprehensive evaluation framework, benchmarking 11 lightweight open-source ViLLMs on CSConDa with both automatic metrics and syntactic analysis to reveal model strengths, weaknesses, and linguistic patterns. This study offers insights into model behavior, explains performance differences, and identifies key areas for improvement, supporting the development of next-generation ViLLMs. By establishing a robust benchmark and systematic evaluation, our work enables informed model selection for customer service QA and advances research on Vietnamese LLMs. The dataset is publicly available at https://huggingface.co/datasets/ura-hcmut/Vietnamese-Customer-Support-QA.