# Pinyin Regularization in Error Correction for Chinese Speech Recognition with Large Language Models

arXiv: https://arxiv.org/abs/2407.01909 · HTML: https://mlsanity.com/paper/2407.01909/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 6 of 6)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario](https://mlsanity.com/paper/2104.03603/) (2021)
- [N-best T5: Robust ASR Error Correction using Multiple Input Hypotheses and Constrained Decoding Space](https://mlsanity.com/paper/2303.00456/) (2023)
- [FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition](https://mlsanity.com/paper/2109.14420/) (2021)
- [Improving Deliberation by Text-Only and Semi-Supervised Training](https://mlsanity.com/paper/2206.14716/) (2022)
- [Generative error correction for code-switching speech recognition using large language models](https://mlsanity.com/paper/2310.13013/) (2023)

## Abstract (arXiv)

Recent studies have demonstrated the efficacy of large language models (LLMs) in error correction for automatic speech recognition (ASR). However, much of the research focuses on the English language. This paper redirects the attention to Chinese. Firstly, we construct a specialized benchmark dataset aimed at error correction for Chinese ASR with 724K hypotheses-transcription pairs, named the Chinese Hypotheses Paradise dataset (ChineseHP), which contains a wide range of scenarios and presents significant challenges. Subsequently, we conduct a preliminary evaluation using the dataset for both direct-prompting and fine-tuning pre-trained LLMs. Furthermore, we propose a straightforward method of Pinyin regularization for prompts, which involves the transcription of Pinyin directly from text hypotheses. The experimental results reveal that Pinyin regularization consistently enhances the error-correcting ability of LLMs when compared with those without regularization. The dataset is available on the website.