Improved Training for End-to-End Streaming Automatic Speech Recognition Model with Punctuation
Not on the map. Why?
Abstract (arXiv)
Punctuated text prediction is crucial for automatic speech recognition as it enhances readability and impacts downstream natural language processing tasks. In streaming scenarios, the ability to predict punctuation in real-time is particularly desirable but presents a difficult technical challenge. In this work, we propose a method for predicting punctuated text from input speech using a chunk-based Transformer encoder trained with Connectionist Temporal Classification (CTC) loss. The acoustic model trained with long sequences by concatenating the input and target sequences can learn punctuation marks attached to the end of sentences more effectively. Additionally, by combining CTC losses on the chunks and utterances, we achieved both the improved F1 score of punctuation prediction and Word Error Rate (WER).
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 19 of its 35 references that are in the map. First posted on arXiv 2 Jun 2023.
Builds on
19 of 35 references.
References · 19
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
- 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
- 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
- 2020Streaming automatic speech recognition with the transformer modelcited by 73
- 2020Streaming Transformer-based Acoustic Models Using Self-attention with Augmented Memorycited by 35
- 2019Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hoppingcited by 33
- 2020Transformer-based Online CTC/attention End-to-End Speech Recognition Architecturecited by 28
- 2020Multimodal Semi-supervised Learning Framework for Punctuation Prediction in Conversational Speechcited by 17
7 more
- 2022E2E Segmenter: Joint Segmenting and Decoding for Long-Form ASRcited by 17
- 2020CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latencycited by 13
- 2014Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognitioncited by 12
- 2020Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognitioncited by 12
- 2022CUSIDE: Chunking, Simulating Future Context and Decoding for Streaming ASRcited by 11
- 2022End-to-end Speech-to-Punctuated-Text Recognitioncited by 7
- 2022Streaming Punctuation for Long-form Dictation with Transformerscited by 4
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.