mlsanityData to 7 Oct 2026

LongFNT: Long-form Speech Recognition with Factorized Neural Transducer

arXiv 2211.09412 ↗First posted 17 Nov 2022cs.SD cs.CL eess.AScited by 6

Not on the map. Why?

Abstract (arXiv)

Traditional automatic speech recognition~(ASR) systems usually focus on individual utterances, without considering long-form speech with useful historical information, which is more practical in real scenarios. Simply attending longer transcription history for a vanilla neural transducer model shows no much gain in our preliminary experiments, since the prediction network is not a pure language model. This motivates us to leverage the factorized neural transducer structure, containing a real language model, the vocabulary predictor. We propose the {LongFNT-Text} architecture, which fuses the sentence-level long-form features directly with the output of the vocabulary predictor and then embeds token-level long-form features inside the vocabulary predictor, with a pre-trained contextual encoder RoBERTa to further boost the performance. Moreover, we propose the {LongFNT} architecture by extending the long-form speech to the original speech input and achieve the best performance. The effectiveness of our LongFNT approach is validated on LibriSpeech and GigaSpeech corpora with 19% and 12% relative word error rate~(WER) reduction, respectively.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 21 of its 36 references that are in the map. First posted on arXiv 17 Nov 2022.

Builds on

21 of 36 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  6. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  7. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  8. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  9. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  10. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  11. 2019Transformer-Transducer: End-to-End Speech Recognition with Self-Attentioncited by 68
  12. 2019A comparison of end-to-end models for long-form speech recognitioncited by 42
9 more
  1. 2021Tied & Reduced RNN-T Decodercited by 35
  2. 2021Factorized Neural Transducer for Efficient Language Model Adaptationcited by 31
  3. 2018Dialog-context aware end-to-end speech recognitioncited by 27
  4. 2021Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformerscited by 25
  5. 2021Hierarchical Transformer-based Large-Context End-to-end ASR with Large-Context Knowledge Distillationcited by 12
  6. 2020Improving RNN-T ASR Accuracy Using Context Audiocited by 8
  7. 2022Conversational Speech Recognition By Learning Conversation-level Characteristicscited by 8
  8. 2022Leveraging Acoustic Contextual Representation by Audio-textual Cross-modal Learning for Conversational ASRcited by 7
  9. 2022Improving Transformer-based Conversational ASR by Inter-Sentential Attention Mechanismcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 1; 2025: 0; 2026: 1.

6 papers.

  1. 2023Updated Corpora and Benchmarks for Long-Form Speech Recognitioncited by 10
  2. 2023Conversational Speech Recognition by Learning Audio-textual Cross-modal Contextual Representationcited by 4
  3. 2026Whisper-CD: Accurate Long-Form Speech Recognition using Multi-Negative Contrastive Decodingcited by 3
  4. 2023Improved Factorized Neural Transducer Model For text-only Domain Adaptationcited by 2
  5. 2024Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASRcited by 2
  6. 2023Hybrid Attention-based Encoder-decoder Model for Efficient Language Model Adaptationcited by 1