mlsanityData to 13 Oct 2026

Capitalization and Punctuation Restoration: a Survey

arXiv 2111.10746 ↗First posted 21 Nov 2021cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Ensuring proper punctuation and letter casing is a key pre-processing step towards applying complex natural language processing algorithms. This is especially significant for textual sources where punctuation and casing are missing, such as the raw output of automatic speech recognition systems. Additionally, short text messages and micro-blogging platforms offer unreliable and often wrong punctuation and casing. This survey offers an overview of both historical and state-of-the-art techniques for restoring punctuation and correcting word casing. Furthermore, current challenges and research directions are highlighted.

Key facts. Cited by 12 papers in the map (data through 13 Oct 2026). Builds on 39 of its 212 references that are in the map. First posted on arXiv 21 Nov 2021.

Builds on

39 of 212 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,539
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,360
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,024
  8. 2015Domain-Adversarial Training of Neural Networkscited by 3,030
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,981
  10. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,910
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,810
  12. 2018Deep contextualized word representationscited by 2,286
27 more
  1. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 769
  2. 2016Neural Architectures for Named Entity Recognitioncited by 549
  3. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 545
  4. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  5. 201975 Languages, 1 Model: Parsing Universal Dependencies Universallycited by 90
  6. 2016Quasi-Recurrent Neural Networkscited by 87
  7. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  8. 2017Massive Exploration of Neural Machine Translation Architecturescited by 55
  9. 2017BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languagescited by 52
  10. 2018Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speechcited by 52
  11. 2015Transition-Based Dependency Parsing with Stack Long Short-Term Memorycited by 45
  12. 2019Cloze-driven Pretraining of Self-attention Networkscited by 41
  13. 2018NCRF++: An Open-source Neural Sequence Labeling Toolkitcited by 29
  14. 2015Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representationcited by 28
  15. 2020Controllable Time-Delay Transformer for Real-Time Punctuation Prediction and Disfluency Detectioncited by 23
  16. 2018Punctuation Prediction Model for Conversational Speechcited by 20
  17. 2020Robust Prediction of Punctuation and Truecasing for Medical ASRcited by 16
  18. 2019ner and pos when nothing is capitalizedcited by 15
  19. 2020Adversarial Transfer Learning for Punctuation Restorationcited by 15
  20. 2019Fast and Accurate Capitalization and Punctuation for Automatic Speech Recognition Using Transformer and Chunk Mergingcited by 14
  21. 2015Larger-Context Language Modellingcited by 10
  22. 2018Mittens: An Extension of GloVe for Learning Domain-Specialized Representationscited by 8
  23. 2020Replacing Human Audio with Synthetic Audio for On-device Unspoken Punctuation Predictioncited by 8
  24. 2017Joint Learning of Correlated Sequence Labelling Tasks Using Bidirectional Recurrent Neural Networkscited by 6
  25. 2020An Efficient Architecture for Predicting the Case of Characters using Sequence Modelscited by 3
  26. 2020Punctuation Prediction in Spontaneous Conversations: Can We Mitigate ASR Errors with Retrofitted Word Embeddings?cited by 3
  27. 2015Transferring Knowledge from a RNN to a DNNcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 6; 2024: 4; 2025: 1.

12 papers.

  1. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 46
  2. 2023Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentationcited by 17
  3. 2024Lyrics Transcription for Humans: A Readability-Aware Benchmarkcited by 5
  4. 2022Streaming Punctuation for Long-form Dictation with Transformerscited by 4
  5. 2023Evaluating OpenAI's Whisper ASR for Punctuation Prediction and Topic Modeling of life histories of the Museum of the Personcited by 4
  6. 2024Punctuation Restoration Improves Structure Understanding Without Supervisioncited by 3
  7. 2023FullStop:Punctuation and Segmentation Prediction for Dutch with Transformerscited by 2
  8. 2023Streaming Punctuation: A Novel Punctuation Technique Leveraging Bidirectional Context for Continuous Speech Recognitioncited by 2
  9. 2023Jam-ALT: A Formatting-Aware Lyrics Transcription Benchmarkcited by 2
  10. 2024Resolving Transcription Ambiguity in Spanish: A Hybrid Acoustic-Lexical System for Punctuation Restorationcited by 2
  11. 2024Beyond Levenshtein: Leveraging Multiple Algorithms for Robust Word Error Rate Computations And Granular Error Classificationscited by 2
  12. 2025Assessing and Improving Punctuation Robustness in English-Marathi Machine Translationcited by 0