# LEGAL-UQA: A Low-Resource Urdu-English Dataset for Legal Question Answering

arXiv: https://arxiv.org/abs/2410.13013 · HTML: https://mlsanity.com/paper/2410.13013/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Builds on (most cited first, 11 of 11)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture](https://mlsanity.com/paper/2306.16092/) (2023)
- [JEC-QA: A Legal-Domain Question Answering Dataset](https://mlsanity.com/paper/1911.12011/) (2019)
- [Modern Question Answering Datasets and Benchmarks: A Survey](https://mlsanity.com/paper/2206.15030/) (2022)
- [UQA: Corpus for Urdu Question Answering](https://mlsanity.com/paper/2405.01458/) (2024)
- [UQuAD1.0: Development of an Urdu Question Answering Training Data for Machine Reading Comprehension](https://mlsanity.com/paper/2111.01543/) (2021)
- [UTRNet: High-Resolution Urdu Text Recognition In Printed Documents](https://mlsanity.com/paper/2306.15782/) (2023)

## Built on by (most cited first, 1 of 1)

- [SinhaLegal: A Benchmark Corpus for Information Extraction and Analysis in Sinhala Legislative Texts](https://mlsanity.com/paper/2603.04854/) (2026)

## Abstract (arXiv)

We present LEGAL-UQA, the first Urdu legal question-answering dataset derived from Pakistan's constitution. This parallel English-Urdu dataset includes 619 question-answer pairs, each with corresponding legal article contexts, addressing the need for domain-specific NLP resources in low-resource languages. We describe the dataset creation process, including OCR extraction, manual refinement, and GPT-4-assisted translation and generation of QA pairs. Our experiments evaluate the latest generalist language and embedding models on LEGAL-UQA, with Claude-3.5-Sonnet achieving 99.19% human-evaluated accuracy. We fine-tune mt5-large-UQA-1.0, highlighting the challenges of adapting multilingual models to specialized domains. Additionally, we assess retrieval performance, finding OpenAI's text-embedding-3-large outperforms Mistral's mistral-embed. LEGAL-UQA bridges the gap between global NLP advancements and localized applications, particularly in constitutional law, and lays the foundation for improved legal information access in Pakistan.