# Transcription Policy as a Latent Variable: Activating Controllable Verbatim ASR with Word-Level Timing

arXiv: https://arxiv.org/abs/2607.18934 · HTML: https://mlsanity.com/paper/2607.18934/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 53 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Builds on (most cited first, 30 of 30)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition](https://mlsanity.com/paper/2205.03433/) (2022)
- [Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and AST](https://mlsanity.com/paper/2509.14128/) (2025)
- [Careless Whisper: Speech-to-Text Hallucination Harms](https://mlsanity.com/paper/2402.08021/) (2024)
- [CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptions](https://mlsanity.com/paper/2408.16589/) (2024)
- [Granary: Speech Recognition and Translation Dataset in 25 European Languages](https://mlsanity.com/paper/2505.13404/) (2025)
- [Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detection](https://mlsanity.com/paper/2312.12810/) (2023)
- [Tradition or Innovation: A Comparison of Modern ASR Methods for Forced Alignment](https://mlsanity.com/paper/2406.19363/) (2024)
- [Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation](https://mlsanity.com/paper/2501.15907/) (2025)
- [From User Perceptions to Technical Improvement: Enabling People Who Stutter to Better Use Speech Recognition](https://mlsanity.com/paper/2302.09044/) (2023)
- [End-to-End Speech Recognition and Disfluency Removal](https://mlsanity.com/paper/2009.10298/) (2020)
- [NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizations](https://mlsanity.com/paper/2508.04195/) (2025)
- [Adapting an ASR Foundation Model for Spoken Language Assessment](https://mlsanity.com/paper/2307.09378/) (2023)
- [Spontaneous Style Text-to-Speech Synthesis with Controllable Spontaneous Behaviors Based on Language Models](https://mlsanity.com/paper/2407.13509/) (2024)
- [Filler Word Detection and Classification: A Dataset and Benchmark](https://mlsanity.com/paper/2203.15135/) (2022)
- [Weakly-supervised forced alignment of disfluent speech using phoneme-level modeling](https://mlsanity.com/paper/2306.00996/) (2023)
- [Streaming Joint Speech Recognition and Disfluency Detection](https://mlsanity.com/paper/2211.08726/) (2022)
- [DisfluencySpeech -- Single-Speaker Conversational Speech Dataset with Paralanguage](https://mlsanity.com/paper/2406.08820/) (2024)
- [Augmenting Automatic Speech Recognition Models with Disfluency Detection](https://mlsanity.com/paper/2409.10177/) (2024)
- [Careful Whisper -- leveraging advances in automatic speech recognition for robust and interpretable aphasia subtype classification](https://mlsanity.com/paper/2308.01327/) (2023)
- [Inclusive ASR for Disfluent Speech: Cascaded Large-Scale Self-Supervised Learning with Targeted Fine-Tuning and Data Augmentation](https://mlsanity.com/paper/2406.10177/) (2024)
- [Style-agnostic evaluation of ASR using multiple reference transcripts](https://mlsanity.com/paper/2412.07937/) (2024)
- [Toward Zero Oracle Word Error Rate on the Switchboard Benchmark](https://mlsanity.com/paper/2206.06192/) (2022)
- [Reverb: Open-Source ASR and Diarization from Rev](https://mlsanity.com/paper/2410.03930/) (2024)
- [Recording for Eyes, Not Echoing to Ears: Contextualized Spoken-to-Written Conversion of ASR Transcripts](https://mlsanity.com/paper/2408.09688/) (2024)
- [Quantification of stylistic differences in human- and ASR-produced transcripts of African American English](https://mlsanity.com/paper/2409.03059/) (2024)
- [Prompting Whisper for Improved Verbatim Transcription and End-to-end Miscue Detection](https://mlsanity.com/paper/2505.23627/) (2025)
- [StutterZero and StutterFormer: End-to-End Speech Conversion for Stuttering Transcription and Correction](https://mlsanity.com/paper/2510.18938/) (2025)

## Built on by (most cited first, 3 of 3)

- [FuseAlign: Forced Alignment in the Wild](https://mlsanity.com/paper/2609.33650/) (2026)
- [InterView-C: A Synchronized Multimodal Corpus of VR Avatar-Mediated Survey Interviews](https://mlsanity.com/paper/2610.10145/) (2026)
- [Where Words Begin and End](https://mlsanity.com/paper/2610.17782/) (2026)

## Abstract (arXiv)

Modern ASR models trained on heterogeneously annotated data treat transcription style (verbatim vs. intended) as an uncontrolled latent variable, causing measurable decoding instability, evaluation confounding (up to 60% of reported WER attributable to style mismatch), and unreliable word-level timing. We show that models already encode both styles; the challenge is controlled activation. Using coverage-aware decoder task tokens trained on parallel verbatim/intended transcript pairs, we raise German disfluency F1 from 10% to 79% zero-shot, despite English-only training. Full English-only fine-tuning surpasses all baselines in verbatim accuracy, disfluency detection, and intended-mode quality across both languages. We further introduce supervised cross-attention fine-tuning that improves word-level timestamps on disfluent speech beyond forced-alignment baselines. Finally, we propose verbatimize, a new task enabling scalable creation and enrichment of speech corpora with high-quality canonical verbatim transcriptions.