# From Speech to Subtitles: Evaluating ASR Models in Subtitling Italian Television Programs

arXiv: https://arxiv.org/abs/2512.19161 · HTML: https://mlsanity.com/paper/2512.19161/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 22 Dec 2025.

## Builds on (most cited first, 11 of 11)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and AST](https://mlsanity.com/paper/2509.14128/) (2025)
- [ASR Error Correction using Large Language Models](https://mlsanity.com/paper/2409.09554/) (2024)
- [Measuring the Accuracy of Automatic Speech Recognition Solutions](https://mlsanity.com/paper/2408.16287/) (2024)
- [Anatomy of Industrial Scale Multilingual ASR](https://mlsanity.com/paper/2404.09841/) (2024)
- [SubER: A Metric for Automatic Evaluation of Subtitle Quality](https://mlsanity.com/paper/2205.05805/) (2022)
- [ASR-EC Benchmark: Evaluating Large Language Models on Chinese ASR Error Correction](https://mlsanity.com/paper/2412.03075/) (2024)
- [Optimizing Estonian TV Subtitles with Semi-supervised Learning and LLMs](https://mlsanity.com/paper/2501.05234/) (2025)

## Abstract (arXiv)

Subtitles are essential for video accessibility and audience engagement. Modern Automatic Speech Recognition (ASR) systems, built upon Encoder-Decoder neural network architectures and trained on massive amounts of data, have progressively reduced transcription errors on standard benchmark datasets. However, their performance in real-world production environments, particularly for non-English content like long-form Italian videos, remains largely unexplored. This paper presents a case study on developing a professional subtitling system for an Italian media company. To inform our system design, we evaluated four state-of-the-art ASR models (Whisper Large v2, AssemblyAI Universal, Parakeet TDT v3 0.6b, and WhisperX) on a 50-hour dataset of Italian television programs. The study highlights their strengths and limitations, benchmarking their performance against the work of professional human subtitlers. The findings indicate that, while current models cannot meet the media industry's accuracy needs for full autonomy, they can serve as highly effective tools for enhancing human productivity. We conclude that a human-in-the-loop (HITL) approach is crucial and present the production-grade, cloud-based infrastructure we designed to support this workflow.