# Corpus Synthesis for Zero-shot ASR domain Adaptation using Large Language Models

arXiv: https://arxiv.org/abs/2309.10707 · HTML: https://mlsanity.com/paper/2309.10707/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 29 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://mlsanity.com/paper/2202.07922/) (2022)
- [Generating Training Data with Language Models: Towards Zero-Shot Language Understanding](https://mlsanity.com/paper/2202.04538/) (2022)
- [SLURP: A Spoken Language Understanding Resource Package](https://mlsanity.com/paper/2011.13205/) (2020)
- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)
- [Locally Typical Sampling](https://mlsanity.com/paper/2202.00666/) (2022)
- [Can Generative Large Language Models Perform ASR Error Correction?](https://mlsanity.com/paper/2307.04172/) (2023)
- [Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition](https://mlsanity.com/paper/2306.16007/) (2023)
- [Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information](https://mlsanity.com/paper/2302.05096/) (2023)
- [Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Network](https://mlsanity.com/paper/2104.11127/) (2021)
- [Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition](https://mlsanity.com/paper/2110.05354/) (2021)
- [Data Selection Strategies for Multi-Domain Sentiment Analysis](https://mlsanity.com/paper/1702.02426/) (2017)
- [Style Equalization: Unsupervised Learning of Controllable Generative Sequence Models](https://mlsanity.com/paper/2110.02891/) (2021)
- [Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generator](https://mlsanity.com/paper/2302.14036/) (2023)
- [Adaptable End-to-End ASR Models using Replaceable Internal LMs and Residual Softmax](https://mlsanity.com/paper/2302.08579/) (2023)
- [A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Data](https://mlsanity.com/paper/2206.13240/) (2022)

## Built on by (most cited first, 5 of 5)

- [Low-Resource Domain Adaptation for Speech LLMs via Text-Only Fine-Tuning](https://mlsanity.com/paper/2506.05671/) (2025)
- [A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Data](https://mlsanity.com/paper/2506.11130/) (2025)
- [Frustratingly Easy Data Augmentation for Low-Resource ASR](https://mlsanity.com/paper/2509.15373/) (2025)
- [Hard-Synth: Synthesizing Diverse Hard Samples for ASR using Zero-Shot TTS and LLM](https://mlsanity.com/paper/2411.13159/) (2024)
- [PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting](https://mlsanity.com/paper/2508.03190/) (2025)

## Abstract (arXiv)

While Automatic Speech Recognition (ASR) systems are widely used in many real-world applications, they often do not generalize well to new domains and need to be finetuned on data from these domains. However, target-domain data usually are not readily available in many scenarios. In this paper, we propose a new strategy for adapting ASR models to new target domains without any text or speech from those domains. To accomplish this, we propose a novel data synthesis pipeline that uses a Large Language Model (LLM) to generate a target domain text corpus, and a state-of-the-art controllable speech synthesis model to generate the corresponding speech. We propose a simple yet effective in-context instruction finetuning strategy to increase the effectiveness of LLM in generating text corpora for new domains. Experiments on the SLURP dataset show that the proposed method achieves an average relative word error rate improvement of 28% on unseen target domains without any performance drop in source domains.