mlsanityData to 14 Oct 2026

Large-scale Language Model Rescoring on Long-form Data

arXiv 2306.08133 ↗First posted 13 Jun 2023eess.AS cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

In this work, we study the impact of Large-scale Language Models (LLM) on Automated Speech Recognition (ASR) of YouTube videos, which we use as a source for long-form ASR. We demonstrate up to 8% relative reduction in Word Error Eate (WER) on US English (en-us) and code-switched Indian English (en-in) long-form ASR test sets and a reduction of up to 30% relative on Salient Term Error Rate (STER) over a strong first-pass baseline that uses a maximum-entropy based language model. Improved lattice processing that results in a lattice with a proper (non-tree) digraph topology and carrying context from the 1-best hypothesis of the previous segment(s) results in significant wins in rescoring with LLMs. We also find that the gains in performance from the combination of LLMs trained on vast quantities of available data (such as C4) and conventional neural LMs is additive and significantly outperforms a strong first-pass baseline with a maximum entropy LM. Copyright 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 13 Jun 2023.

Builds on

24 of 38 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  8. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  9. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 185
  10. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 150
  11. 2017An analysis of incorporating an external language model into a sequence-to-sequence modelcited by 111
  12. 2020Hybrid Autoregressive Transducer (hat)cited by 108
12 more
  1. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  2. 2017Cold Fusion: Training Seq2Seq Models Together with Language Modelscited by 88
  3. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  4. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  5. 2021Tied & Reduced RNN-T Decodercited by 36
  6. 2020Less Is More: Improved RNN-T Decoding Using Limited Label Context and Path Mergingcited by 26
  7. 2020Improving Tail Performance of a Deliberation E2E ASR Model Using a Large Text Corpuscited by 24
  8. 2018End-to-end Speech Recognition with Word-based RNN Language Modelscited by 23
  9. 2021Innovative Bert-based Reranking Language Models for Speech Recognitioncited by 23
  10. 2021Adapting GPT, GPT-2 and BERT Language Models for Speech Recognitioncited by 16
  11. 2022Improving Deliberation by Text-Only and Semi-Supervised Trainingcited by 10
  12. 2020Improving accuracy of rare words for RNN-Transducer through unigram shallow fusioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 3; 2026: 1.

14 papers.

Sort:

By area:

  1. 2023SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translationcited by 76
  2. 2023Multi-stage Large Language Model Correction for Speech Recognitioncited by 13
  3. 2024Multilingual and Fully Non-Autoregressive ASR with Large Language Model Fusion: A Comprehensive Studycited by 12
  4. 2023Updated Corpora and Benchmarks for Long-Form Speech Recognitioncited by 10
  5. 2025When Large Language Models Meet Speech: A Survey on Integration Approachescited by 9
  6. 2023Contextual Biasing of Named-Entities with Large Language Modelscited by 7
  7. 2024KVPruner: Structural Pruning for Faster and Memory-Efficient Large Language Modelscited by 6
  8. 2024Decoding the Diversity: A Review of the Indic AI Research Landscapecited by 4
  9. 2025Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Reviewcited by 4
  10. 2023Correction Focused Language Model Training for Speech Recognitioncited by 2
  11. 2024Speech Recognition With LLMs Adapted to Disordered Speech Using Reinforcement Learningcited by 2
  12. 2024Speech Prefix-Tuning with RNNT Loss for Improving LLM Predictionscited by 1
2 more
  1. 2025Customizing Speech Recognition Model with Large Language Model Feedbackcited by 1
  2. 2026Merging the Knowledge of LLMs for Automatic Speech Recognitioncited by 0