# Generative error correction for code-switching speech recognition using large language models

arXiv: https://arxiv.org/abs/2310.13013 · HTML: https://mlsanity.com/paper/2310.13013/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 13 of 13)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Multilingual and code-switching ASR challenges for low resource Indian languages](https://mlsanity.com/paper/2104.00235/) (2021)
- [The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Results](https://mlsanity.com/paper/2007.05916/) (2020)
- [Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization](https://mlsanity.com/paper/2305.11095/) (2023)
- [HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models](https://mlsanity.com/paper/2309.15701/) (2023)
- [Reducing Language confusion for Code-switching Speech Recognition with Token-level Language Diarization](https://mlsanity.com/paper/2210.14567/) (2022)
- [Towards End-to-End Code-Switching Speech Recognition](https://mlsanity.com/paper/1810.13091/) (2018)
- [End-to-End Code-Switching ASR for Low-Resourced Language Pairs](https://mlsanity.com/paper/1909.12681/) (2019)
- [Towards Zero-Shot Code-Switched Speech Recognition](https://mlsanity.com/paper/2211.01458/) (2022)
- [Internal Language Model Estimation based Language Model Fusion for Cross-Domain Code-Switching Speech Recognition](https://mlsanity.com/paper/2207.04176/) (2022)

## Built on by (most cited first, 7 of 7)

- [Large Language Models are Efficient Learners of Noise-Robust Speech Recognition](https://mlsanity.com/paper/2401.10446/) (2024)
- [It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition](https://mlsanity.com/paper/2402.05457/) (2024)
- [LLM-based Generative Error Correction for Rare Words with Synthetic Data and Phonetic Context](https://mlsanity.com/paper/2505.17410/) (2025)
- [Full-text Error Correction for Chinese Speech Recognition with Large Language Model](https://mlsanity.com/paper/2409.07790/) (2024)
- [Chain of Correction for Full-text Speech Recognition with Large Language Models](https://mlsanity.com/paper/2504.01519/) (2025)
- [Pinyin Regularization in Error Correction for Chinese Speech Recognition with Large Language Models](https://mlsanity.com/paper/2407.01909/) (2024)
- [Speech Recognition on TV Series with Video-guided Post-ASR Correction](https://mlsanity.com/paper/2506.07323/) (2025)

## Abstract (arXiv)

Code-switching (CS) speech refers to the phenomenon of mixing two or more languages within the same sentence. Despite the recent advances in automatic speech recognition (ASR), CS-ASR is still a challenging task ought to the grammatical structure complexity of the phenomenon and the data scarcity of specific training corpus. In this work, we propose to leverage large language models (LLMs) and lists of hypotheses generated by an ASR to address the CS problem. Specifically, we first employ multiple well-trained ASR models for N-best hypotheses generation, with the aim of increasing the diverse and informative elements in the set of hypotheses. Next, we utilize the LLMs to learn the hypotheses-to-transcription (H2T) mapping by adding a trainable low-rank adapter. Such a generative error correction (GER) method directly predicts the accurate transcription according to its expert linguistic knowledge and N-best hypotheses, resulting in a paradigm shift from the traditional language model rescoring or error correction techniques. Experimental evidence demonstrates that GER significantly enhances CS-ASR accuracy, in terms of reduced mixed error rate (MER). Furthermore, LLMs show remarkable data efficiency for H2T learning, providing a potential solution to the data scarcity problem of CS-ASR in low-resource languages.