mlsanityData to 7 Oct 2026

Adapting Small Language Models to Low-Resource Domains: A Case Study in Hindi Tourism QA

arXiv 2510.25273 ↗First posted 29 Oct 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Domain-specific question answering in low-resource languages faces two key challenges: scarcity of annotated datasets and limited domain knowledge in general-purpose language models. In this work, we present a multi-stage finetuning strategy to adapt lightweight language models to the Hindi tourism domain by leveraging both original and synthetic training data. Synthetic question-answer pairs are generated using large LLMs (LLaMA-70B, Phi-14B) and used to augment the limited original dataset. We explore several training methodologies and analyse their impact on domain generalisation. Our results demonstrate that large models can efficiently generate synthetic data, while small models can effectively adapt to it, offering a scalable pathway for low-resource, domain-specific QA.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 22 references that are in the map. First posted on arXiv 29 Oct 2025.

Builds on

14 of 22 references.

References · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  3. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
  4. 2021Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA?cited by 62
  5. 2022RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extractioncited by 60
  6. 2021FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text modelscited by 26
  7. 2024INDIC QA BENCHMARK: A Multilingual Benchmark to Evaluate Question Answering capability of LLMs for Indic Languagescited by 24
  8. 2022Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generationcited by 17
  9. 2024Adapting Multilingual LLMs to Low-Resource Languages with Knowledge Graphs via Adapterscited by 14
  10. 2023MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answeringcited by 12
  11. 2024Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Promptingcited by 10
  12. 2024Prompting-based Synthetic Data Generation for Few-Shot Question Answeringcited by 9
2 more
  1. 2023Generative Data Augmentation using LLMs improves Distributional Robustness in Question Answeringcited by 5
  2. 2024GeMQuAD : Generating Multilingual Question Answering Datasets from Large Language Models using Few Shot Learningcited by 3

Built on by

Not cited yet.