Factual Consistency Oriented Speech Recognition
Not on the map. Why?
Abstract (arXiv)
This paper presents a novel optimization framework for automatic speech recognition (ASR) with the aim of reducing hallucinations produced by an ASR model. The proposed framework optimizes the ASR model to maximize an expected factual consistency score between ASR hypotheses and ground-truth transcriptions, where the factual consistency score is computed by a separately trained estimator. Experimental results using the AMI meeting corpus and the VoxPopuli corpus show that the ASR model trained with the proposed framework generates ASR hypotheses that have significantly higher consistency scores with ground-truth transcriptions while maintaining the word error rates close to those of cross entropy-trained ASR models. Furthermore, it is shown that training the ASR models with the proposed framework improves the speech summarization quality as measured by the factual consistency of meeting conversation summaries generated by a large language model.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 45 references that are in the map. First posted on arXiv 24 Feb 2023.
Builds on
28 of 45 references.
References · 28
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
- 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
- 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 449
- 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
16 more
- 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
- 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
- 2018How2: A Large-scale Dataset for Multimodal Language Understandingcited by 137
- 2020Factual Error Correction for Abstractive Summarization Modelscited by 108
- 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
- 2014End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Resultscited by 68
- 2017Minimum Word Error Rate Training for Attention-based Sequence-to-Sequence Modelscited by 68
- 2016Achieving Human Parity in Conversational Speech Recognitioncited by 63
- 2018From Audio to Semantics: Approaches to end-to-end spoken language understandingcited by 63
- 2022SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Taskscited by 34
- 2021Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphonecited by 33
- 2019Lessons from Building Acoustic Models with a Million Hours of Speechcited by 27
- 2020Speech To Semantics: Improve ASR and NLU Jointly via All-Neural Interfacescited by 24
- 2021Speech Summarization using Restricted Self-Attentioncited by 23
- 2017Optimizing expected word error rate via sampling for speech recognitioncited by 21
- 2021Do as I mean, not as I say: Sequence Loss Training for Spoken Language Understandingcited by 9
Built on by
Citing papers per year: 2025: 1.
1 paper.