mlsanityData to 7 Oct 2026

Automatic punctuation restoration with BERT models

arXiv 2101.07343 ↗First posted 18 Jan 2021cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

We present an approach for automatic punctuation restoration with BERT models for English and Hungarian. For English, we conduct our experiments on Ted Talks, a commonly used benchmark for punctuation restoration, while for Hungarian we evaluate our models on the Szeged Treebank dataset. Our best models achieve a macro-averaged F1-score of 79.8 in English and 82.2 in Hungarian. Our code is publicly available.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 6 of its 23 references that are in the map. First posted on arXiv 18 Jan 2021.

Builds on

6 of 23 references.

References · 6

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2020Controllable Time-Delay Transformer for Real-Time Punctuation Prediction and Disfluency Detectioncited by 23
  5. 2018Punctuation Prediction Model for Conversational Speechcited by 20
  6. 2019Fast and Accurate Capitalization and Punctuation for Automatic Speech Recognition Using Transformer and Chunk Mergingcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 3; 2023: 3; 2024: 1; 2025: 1; 2026: 1.

12 papers.

  1. 2021Token-Level Supervised Contrastive Learning for Punctuation Restorationcited by 14
  2. 2021Improving Punctuation Restoration for Speech Transcripts via External Datacited by 8
  3. 2022Streaming Punctuation for Long-form Dictation with Transformerscited by 4
  4. 2023Streaming Punctuation: A Novel Punctuation Technique Leveraging Bidirectional Context for Continuous Speech Recognitioncited by 2
  5. 2021ViraPart: A Text Refinement Framework for Automatic Speech Recognition and Natural Language Processing Tasks in Persiancited by 1
  6. 2026PersianPunc: A Large-Scale Dataset and BERT-Based Approach for Persian Punctuation Restorationcited by 1
  7. 2022Punctuation restoration in Swedish through fine-tuned KB-BERTcited by 0
  8. 2022Punctuation Restoration for Singaporean Spoken Languages: English, Malay, and Mandarincited by 0
  9. 2023Punctuation restoration Model and Spacing Model for Korean Ancient Documentcited by 0
  10. 2023Beyond the Frame: Single and mutilple video summarization method with user-defined lengthcited by 0
  11. 2024A light-weight and efficient punctuation and word casing prediction model for on-device streaming ASRcited by 0
  12. 2025Assessing and Improving Punctuation Robustness in English-Marathi Machine Translationcited by 0