# Adapting End-to-End Speech Recognition for Readable Subtitles

arXiv: https://arxiv.org/abs/2005.12143 · HTML: https://mlsanity.com/paper/2005.12143/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2020.

## Built on by (most cited first, 10 of 10)

- [Isometric MT: Neural Machine Translation for Automatic Dubbing](https://mlsanity.com/paper/2112.08682/) (2021)
- [Direct Speech Translation for Automatic Subtitling](https://mlsanity.com/paper/2209.13192/) (2022)
- [Joint Generation of Captions and Subtitles with Dual Decoding](https://mlsanity.com/paper/2205.06522/) (2022)
- [Summarizing Speech: A Comprehensive Survey](https://mlsanity.com/paper/2504.08024/) (2025)
- [Evaluating Subtitle Segmentation for End-to-end Generation Systems](https://mlsanity.com/paper/2205.09360/) (2022)
- [AppTek's Submission to the IWSLT 2022 Isometric Spoken Language Translation Task](https://mlsanity.com/paper/2205.05807/) (2022)
- [Sentence-wise Speech Summarization: Task, Datasets, and End-to-End Modeling with LM Knowledge Distillation](https://mlsanity.com/paper/2408.00205/) (2024)
- [Learning to Jointly Transcribe and Subtitle for End-to-End Spontaneous Speech Recognition](https://mlsanity.com/paper/2210.07771/) (2022)
- [Re-evaluating Minimum Bayes Risk Decoding for Automatic Speech Recognition](https://mlsanity.com/paper/2510.19471/) (2025)
- [Length Aware Speech Translation for Video Dubbing](https://mlsanity.com/paper/2506.00740/) (2025)

## Abstract (arXiv)

Automatic speech recognition (ASR) systems are primarily evaluated on transcription accuracy. However, in some use cases such as subtitling, verbatim transcription would reduce output readability given limited screen size and reading time. Therefore, this work focuses on ASR with output compression, a task challenging for supervised approaches due to the scarcity of training data. We first investigate a cascaded system, where an unsupervised compression model is used to post-edit the transcribed speech. We then compare several methods of end-to-end speech recognition under output length constraints. The experiments show that with limited data far less than needed for training a model from scratch, we can adapt a Transformer-based ASR model to incorporate both transcription and compression capabilities. Furthermore, the best performance in terms of WER and ROUGE scores is achieved by explicitly modeling the length constraints within the end-to-end ASR system.