# Streaming Punctuation for Long-form Dictation with Transformers

arXiv: https://arxiv.org/abs/2210.05756 · HTML: https://mlsanity.com/paper/2210.05756/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 28 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [Punctuation Prediction Model for Conversational Speech](https://mlsanity.com/paper/1807.00543/) (2018)
- [Capitalization and Punctuation Restoration: a Survey](https://mlsanity.com/paper/2111.10746/) (2021)
- [Automatic punctuation restoration with BERT models](https://mlsanity.com/paper/2101.07343/) (2021)
- [Joint prediction of truecasing and punctuation for conversational speech in low-resource scenarios](https://mlsanity.com/paper/2109.06103/) (2021)
- [A Context-Aware Feature Fusion Framework for Punctuation Restoration](https://mlsanity.com/paper/2203.12487/) (2022)

## Built on by (most cited first, 4 of 4)

- [Smart Speech Segmentation using Acousto-Linguistic Features with look-ahead](https://mlsanity.com/paper/2210.14446/) (2022)
- [Improved Training for End-to-End Streaming Automatic Speech Recognition Model with Punctuation](https://mlsanity.com/paper/2306.01296/) (2023)
- [Streaming Punctuation: A Novel Punctuation Technique Leveraging Bidirectional Context for Continuous Speech Recognition](https://mlsanity.com/paper/2301.03819/) (2023)
- [TRScore: A Novel GPT-based Readability Scorer for ASR Segmentation and Punctuation model evaluation and selection](https://mlsanity.com/paper/2210.15104/) (2022)

## Abstract (arXiv)

While speech recognition Word Error Rate (WER) has reached human parity for English, long-form dictation scenarios still suffer from segmentation and punctuation problems resulting from irregular pausing patterns or slow speakers. Transformer sequence tagging models are effective at capturing long bi-directional context, which is crucial for automatic punctuation. Automatic Speech Recognition (ASR) production systems, however, are constrained by real-time requirements, making it hard to incorporate the right context when making punctuation decisions. In this paper, we propose a streaming approach for punctuation or re-punctuation of ASR output using dynamic decoding windows and measure its impact on punctuation and segmentation accuracy across scenarios. The new system tackles over-segmentation issues, improving segmentation F0.5-score by 13.9%. Streaming punctuation achieves an average BLEU-score improvement of 0.66 for the downstream task of Machine Translation (MT).