mlsanityData to 14 Oct 2026

Transcription Policy as a Latent Variable: Activating Controllable Verbatim ASR with Word-Level Timing

arXiv 2607.18934 ↗First posted 21 Jul 2026cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Modern ASR models trained on heterogeneously annotated data treat transcription style (verbatim vs. intended) as an uncontrolled latent variable, causing measurable decoding instability, evaluation confounding (up to 60% of reported WER attributable to style mismatch), and unreliable word-level timing. We show that models already encode both styles; the challenge is controlled activation. Using coverage-aware decoder task tokens trained on parallel verbatim/intended transcript pairs, we raise German disfluency F1 from 10% to 79% zero-shot, despite English-only training. Full English-only fine-tuning surpasses all baselines in verbatim accuracy, disfluency detection, and intended-mode quality across both languages. We further introduce supervised cross-attention fine-tuning that improves word-level timestamps on disfluent speech beyond forced-alignment baselines. Finally, we propose verbatimize, a new task enabling scalable creation and enrichment of speech corpora with high-quality canonical verbatim transcriptions.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 30 of its 53 references that are in the map. First posted on arXiv 21 Jul 2026.

Builds on

30 of 53 references.

References · 30

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  3. 2023WhisperX: Time-Accurate Speech Transcription of Long-Form Audiocited by 326
  4. 2022Vocalsound: A Dataset for Improving Human Vocal Sounds Recognitioncited by 62
  5. 2025Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and ASTcited by 60
  6. 2024Careless Whisper: Speech-to-Text Hallucination Harmscited by 53
  7. 2024CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptionscited by 34
  8. 2025Granary: Speech Recognition and Translation Dataset in 25 European Languagescited by 22
  9. 2023Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detectioncited by 21
  10. 2024Tradition or Innovation: A Comparison of Modern ASR Methods for Forced Alignmentcited by 21
  11. 2025Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generationcited by 20
  12. 2023From User Perceptions to Technical Improvement: Enabling People Who Stutter to Better Use Speech Recognitioncited by 18
18 more
  1. 2020End-to-End Speech Recognition and Disfluency Removalcited by 16
  2. 2025NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizationscited by 15
  3. 2023Adapting an ASR Foundation Model for Spoken Language Assessmentcited by 14
  4. 2024Spontaneous Style Text-to-Speech Synthesis with Controllable Spontaneous Behaviors Based on Language Modelscited by 8
  5. 2022Filler Word Detection and Classification: A Dataset and Benchmarkcited by 7
  6. 2023Weakly-supervised forced alignment of disfluent speech using phoneme-level modelingcited by 7
  7. 2022Streaming Joint Speech Recognition and Disfluency Detectioncited by 6
  8. 2024DisfluencySpeech -- Single-Speaker Conversational Speech Dataset with Paralanguagecited by 6
  9. 2024Augmenting Automatic Speech Recognition Models with Disfluency Detectioncited by 6
  10. 2023Careful Whisper -- leveraging advances in automatic speech recognition for robust and interpretable aphasia subtype classificationcited by 5
  11. 2024Inclusive ASR for Disfluent Speech: Cascaded Large-Scale Self-Supervised Learning with Targeted Fine-Tuning and Data Augmentationcited by 5
  12. 2024Style-agnostic evaluation of ASR using multiple reference transcriptscited by 5
  13. 2022Toward Zero Oracle Word Error Rate on the Switchboard Benchmarkcited by 4
  14. 2024Reverb: Open-Source ASR and Diarization from Revcited by 3
  15. 2024Recording for Eyes, Not Echoing to Ears: Contextualized Spoken-to-Written Conversion of ASR Transcriptscited by 2
  16. 2024Quantification of stylistic differences in human- and ASR-produced transcripts of African American Englishcited by 2
  17. 2025Prompting Whisper for Improved Verbatim Transcription and End-to-end Miscue Detectioncited by 1
  18. 2025StutterZero and StutterFormer: End-to-End Speech Conversion for Stuttering Transcription and Correctioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.