# LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Models

arXiv: https://arxiv.org/abs/2310.02943 · HTML: https://mlsanity.com/paper/2310.02943/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus](https://mlsanity.com/paper/2105.02732/) (2021)
- [Punctuation Prediction Model for Conversational Speech](https://mlsanity.com/paper/1807.00543/) (2018)
- [Multimodal Semi-supervised Learning Framework for Punctuation Prediction in Conversational Speech](https://mlsanity.com/paper/2008.00702/) (2020)
- [Fast and Accurate Capitalization and Punctuation for Automatic Speech Recognition Using Transformer and Chunk Merging](https://mlsanity.com/paper/1908.02404/) (2019)
- [Capitalization and Punctuation Restoration: a Survey](https://mlsanity.com/paper/2111.10746/) (2021)
- [Golos: Russian Dataset for Speech Research](https://mlsanity.com/paper/2106.10161/) (2021)
- [End-to-end Speech-to-Punctuated-Text Recognition](https://mlsanity.com/paper/2207.03169/) (2022)
- [End to End ASR System with Automatic Punctuation Insertion](https://mlsanity.com/paper/2012.02012/) (2020)

## Built on by (most cited first, 30 of 49)

- [F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching](https://mlsanity.com/paper/2410.06885/) (2024)
- [DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation](https://mlsanity.com/paper/2502.03930/) (2025)
- [ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching](https://mlsanity.com/paper/2506.13053/) (2025)
- [OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models](https://mlsanity.com/paper/2604.00688/) (2026)
- [Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASR](https://mlsanity.com/paper/2409.15869/) (2024)
- [PhaseCoder: Microphone Geometry-Agnostic Spatial Audio Understanding for Multimodal LLMs](https://mlsanity.com/paper/2601.21124/) (2026)
- [The Design Space of Tri-Modal Masked Diffusion Models](https://mlsanity.com/paper/2602.21472/) (2026)
- [DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generation](https://mlsanity.com/paper/2503.22265/) (2025)
- [Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling](https://mlsanity.com/paper/2505.19931/) (2025)
- [Cross-Lingual F5-TTS: Towards Language-Agnostic Voice Cloning and Speech Synthesis](https://mlsanity.com/paper/2509.14579/) (2025)
- [Longer is (Not Necessarily) Stronger: Punctuated Long-Sequence Training for Enhanced Speech Recognition and Translation](https://mlsanity.com/paper/2409.05601/) (2024)
- [Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignment](https://mlsanity.com/paper/2505.19595/) (2025)
- [Flow2GAN: Hybrid Flow Matching and GAN with Multi-Resolution Network for Few-step High-Fidelity Audio Generation](https://mlsanity.com/paper/2512.23278/) (2025)
- [DiSTAR: Diffusion over a Scalable Token Autoregressive Representation for Speech Generation](https://mlsanity.com/paper/2510.12210/) (2025)
- [VoXtream2: Full-stream TTS with dynamic speaking rate control](https://mlsanity.com/paper/2603.13518/) (2026)
- [Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTS](https://mlsanity.com/paper/2605.30748/) (2026)
- [Beyond Levenshtein: Leveraging Multiple Algorithms for Robust Word Error Rate Computations And Granular Error Classifications](https://mlsanity.com/paper/2408.15616/) (2024)
- [SimulMEGA: MoE Routers are Advanced Policy Makers for Simultaneous Speech Translation](https://mlsanity.com/paper/2509.01200/) (2025)
- [Length-Aware Rotary Position Embedding for Text-Speech Alignment](https://mlsanity.com/paper/2509.11084/) (2025)
- [Unifying Speech Recognition, Synthesis and Conversion with Autoregressive Transformers](https://mlsanity.com/paper/2601.10770/) (2026)
- [Minimizing Modality Gap from the Input Side: Your Speech LLM Can Be a Prosody-Aware Text LLM](https://mlsanity.com/paper/2605.05927/) (2026)
- [End-to-end Joint Punctuated and Normalized ASR with a Limited Amount of Punctuated Training Data](https://mlsanity.com/paper/2311.17741/) (2023)
- [Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space](https://mlsanity.com/paper/2505.13181/) (2025)
- [Computational Narrative Understanding for Expressive Text-to-Speech](https://mlsanity.com/paper/2509.04072/) (2025)
- [DiTReducio: A Training-Free Acceleration for DiT-Based TTS via Progressive Calibration](https://mlsanity.com/paper/2509.09748/) (2025)
- [Vox-Evaluator: Enhancing Stability and Fidelity for Zero-shot TTS with A Multi-Level Evaluator](https://mlsanity.com/paper/2510.20210/) (2025)
- [ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference](https://mlsanity.com/paper/2602.05207/) (2026)
- [WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling](https://mlsanity.com/paper/2606.03455/) (2026)
- [Transcript-Free Flow-Matching Text-to-Speech via Speech Feature Conditioning](https://mlsanity.com/paper/2606.20266/) (2026)
- [DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech](https://mlsanity.com/paper/2607.04140/) (2026)

## Abstract (arXiv)

Traditional automatic speech recognition (ASR) models output lower-cased words without punctuation marks, which reduces readability and necessitates a subsequent text processing model to convert ASR transcripts into a proper format. Simultaneously, the development of end-to-end ASR models capable of predicting punctuation and capitalization presents several challenges, primarily due to limited data availability and shortcomings in the existing evaluation methods, such as inadequate assessment of punctuation prediction. In this paper, we introduce a LibriSpeech-PC benchmark designed to assess the punctuation and capitalization prediction capabilities of end-to-end ASR models. The benchmark includes a LibriSpeech-PC dataset with restored punctuation and capitalization, a novel evaluation metric called Punctuation Error Rate (PER) that focuses on punctuation marks, and initial baseline models. All code, data, and models are publicly available.