mlsanityData to 7 Oct 2026

LLM-based Generative Error Correction for Rare Words with Synthetic Data and Phonetic Context

arXiv 2505.17410 ↗First posted 23 May 2025cs.SD cs.CL eess.AScited by 3

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

Generative error correction (GER) with large language models (LLMs) has emerged as an effective post-processing approach to improve automatic speech recognition (ASR) performance. However, it often struggles with rare or domain-specific words due to limited training data. Furthermore, existing LLM-based GER approaches primarily rely on textual information, neglecting phonetic cues, which leads to over-correction. To address these issues, we propose a novel LLM-based GER approach that targets rare words and incorporates phonetic information. First, we generate synthetic data to contain rare words for fine-tuning the GER model. Second, we integrate ASR's N-best hypotheses along with phonetic context to mitigate over-correction. Experimental results show that our method not only improves the correction of rare words but also reduces the WER and CER across both English and Japanese datasets.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 28 of its 39 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

28 of 39 references.

References · 28

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  6. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  7. 2023End-to-End Speech Recognition: A Surveycited by 127
  8. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 100
  9. 2020Spelling Error Correction with Soft-Masked BERTcited by 90
  10. 2023Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translationcited by 74
  11. 2023Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Promptingcited by 71
  12. 2017JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesiscited by 57
16 more
  1. 2023Can Generative Large Language Models Perform ASR Error Correction?cited by 56
  2. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 41
  3. 2020Distilling the Knowledge of BERT for Sequence-to-Sequence ASRcited by 38
  4. 2021EDGAR-CORPUS: Billions of Tokens Make The World Go Roundcited by 32
  5. 2021Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modelingcited by 31
  6. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  7. 2023Multi-stage Large Language Model Correction for Speech Recognitioncited by 13
  8. 2024Retrieval Augmented Correction of Named Entity Speech Recognition Errorscited by 10
  9. 2024LLM-Powered Grapheme-to-Phoneme Conversion: Benchmark and Case Studycited by 9
  10. 2023ed-cec: improving rare word recognition using asr postprocessing based on error detection and context-aware error correctioncited by 7
  11. 2023Generative error correction for code-switching speech recognition using large language modelscited by 7
  12. 2022Non-autoregressive Error Correction for CTC-based ASR with Phone-conditioned Masked LMcited by 6
  13. 2024InterBiasing: Boost Unseen Word Recognition through Biasing Intermediate Predictionscited by 5
  14. 2024Benchmarking Japanese Speech Recognition on ASR-LLM Setups with Multi-Pass Augmented Generative Error Correctioncited by 4
  15. 2024Data-driven grapheme-to-phoneme representations for a lexicon-free text-to-speechcited by 3
  16. 2024Can large language models understand uncommon meanings of common words?cited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Knowledge-Augmented Question Error Correction for Chinese Question Answer System with QuestionRAGcited by 0
  2. 2026Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisiancited by 0
  3. 2026Massive Open-Vocabulary Keyword Spottingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Models
  2. 2024Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentation
  3. 2024Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
  4. 2024Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition
  5. 2025GEC-RAG: Improving Generative Error Correction via Retrieval-Augmented Generation for Automatic Speech Recognition Systems
  6. 2026agentic-ger: terminology recovery in long-form speech using global context
  7. 2025DeRAGEC: Denoising Named Entity Candidates with Synthetic Rationale for ASR Error Correction
  8. 2025Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correction