# Improving Vietnamese Legal Document Retrieval using Synthetic Data

arXiv: https://arxiv.org/abs/2412.00657 · HTML: https://mlsanity.com/paper/2412.00657/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 1 Dec 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [LLM synthetic data generation](https://mlsanity.com/synth/llm-synthetic-data-generation/)

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Textbooks Are All You Need](https://mlsanity.com/paper/2306.11644/) (2023)
- [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://mlsanity.com/paper/2306.02707/) (2023)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [Embedding-based Zero-shot Retrieval through Query Generation](https://mlsanity.com/paper/2009.10270/) (2020)
- [Multi-stage Information Retrieval for Vietnamese Legal Texts](https://mlsanity.com/paper/2209.14494/) (2022)
- [Towards Comprehensive Vietnamese Retrieval-Augmented Generation and Large Language Models](https://mlsanity.com/paper/2403.01616/) (2024)

## Built on by (most cited first, 1 of 1)

- [Legal Document Summarization: Enhancing Judicial Efficiency through Automation Detection](https://mlsanity.com/paper/2507.18952/) (2025)

## Abstract (arXiv)

In the field of legal information retrieval, effective embedding-based models are essential for accurate question-answering systems. However, the scarcity of large annotated datasets poses a significant challenge, particularly for Vietnamese legal texts. To address this issue, we propose a novel approach that leverages large language models to generate high-quality, diverse synthetic queries for Vietnamese legal passages. This synthetic data is then used to pre-train retrieval models, specifically bi-encoder and ColBERT, which are further fine-tuned using contrastive loss with mined hard negatives. Our experiments demonstrate that these enhancements lead to strong improvement in retrieval accuracy, validating the effectiveness of synthetic data and pre-training techniques in overcoming the limitations posed by the lack of large labeled datasets in the Vietnamese legal domain.