# Fast and Accurate Capitalization and Punctuation for Automatic Speech Recognition Using Transformer and Chunk Merging

arXiv: https://arxiv.org/abs/1908.02404 · HTML: https://mlsanity.com/paper/1908.02404/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2019.

## Built on by (most cited first, 14 of 14)

- [SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition](https://mlsanity.com/paper/2104.02014/) (2021)
- [LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Models](https://mlsanity.com/paper/2310.02943/) (2023)
- [BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese](https://mlsanity.com/paper/2109.09701/) (2021)
- [Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentation](https://mlsanity.com/paper/2305.18893/) (2023)
- [Capitalization and Punctuation Restoration: a Survey](https://mlsanity.com/paper/2111.10746/) (2021)
- [Automatic punctuation restoration with BERT models](https://mlsanity.com/paper/2101.07343/) (2021)
- [Discriminative Self-training for Punctuation Prediction](https://mlsanity.com/paper/2104.10339/) (2021)
- [Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model](https://mlsanity.com/paper/2202.08171/) (2022)
- [Position-Invariant Truecasing with a Word-and-Character Hierarchical Recurrent Neural Network](https://mlsanity.com/paper/2108.11943/) (2021)
- [Vietnamese Capitalization and Punctuation Recovery Models](https://mlsanity.com/paper/2207.01312/) (2022)
- [Text Injection for Capitalization and Turn-Taking Prediction in Speech Models](https://mlsanity.com/paper/2308.07395/) (2023)
- [End-to-end Joint Punctuated and Normalized ASR with a Limited Amount of Punctuated Training Data](https://mlsanity.com/paper/2311.17741/) (2023)
- [Zero-Shot Joint Modeling of Multiple Spoken-Text-Style Conversion Tasks using Switching Tokens](https://mlsanity.com/paper/2106.12131/) (2021)
- [Mask-combine Decoding and Classification Approach for Punctuation Prediction with real-time Inference Constraints](https://mlsanity.com/paper/2112.08098/) (2021)

## Abstract (arXiv)

In recent years, studies on automatic speech recognition (ASR) have shown outstanding results that reach human parity on short speech segments. However, there are still difficulties in standardizing the output of ASR such as capitalization and punctuation restoration for long-speech transcription. The problems obstruct readers to understand the ASR output semantically and also cause difficulties for natural language processing models such as NER, POS and semantic parsing. In this paper, we propose a method to restore the punctuation and capitalization for long-speech ASR transcription. The method is based on Transformer models and chunk merging that allows us to (1), build a single model that performs punctuation and capitalization in one go, and (2), perform decoding in parallel while improving the prediction accuracy. Experiments on British National Corpus showed that the proposed approach outperforms existing methods in both accuracy and decoding speed.