Mark My Words: A Robust Multilingual Model for Punctuation in Text and Speech Transcripts
Not on the map. Why?
Abstract (arXiv)
Punctuation plays a vital role in structuring meaning, yet current models often struggle to restore it accurately in transcripts of spontaneous speech, especially in the presence of disfluencies such as false starts and backtracking. These limitations hinder the performance of downstream tasks like translation, text to speech, summarization, etc. where sentence boundaries are critical for preserving quality. In this work, we introduce Cadence, a generalist punctuation restoration model adapted from a pretrained large language model. Cadence is designed to handle both clean written text and highly spontaneous spoken transcripts. It surpasses the previous state of the art in performance while expanding support from 14 to all 22 Indian languages and English. We conduct a comprehensive analysis of model behavior across punctuation types and language families, identifying persistent challenges under domain shift and with rare punctuation marks. Our findings demonstrate the efficacy of utilizing pretrained language models for multilingual punctuation restoration and highlight Cadence practical value for low resource NLP pipelines at scale.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 12 of its 24 references that are in the map. First posted on arXiv 4 Jun 2025.
Builds on
12 of 24 references.
References · 12
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
- 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
- 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
- 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
- 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 152
- 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 102
- 2024IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languagescited by 52
- 2024IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languagescited by 41
- 2022IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languagescited by 27
- 2022indic-punct: An automatic punctuation restoration and inverse text normalization framework for Indic languagescited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.