mlsanityData to 14 Oct 2026

Speech Recognition on TV Series with Video-guided Post-ASR Correction

arXiv 2506.07323 ↗First posted 8 Jun 2025cs.SD cs.AI eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Automatic Speech Recognition (ASR) has achieved remarkable success with deep learning, driving advancements in conversational artificial intelligence, media transcription, and assistive technologies. However, ASR systems still struggle in complex environments such as TV series, where multiple speakers, overlapping speech, domain-specific terminology, and long-range contextual dependencies pose significant challenges to transcription accuracy. Existing approaches fail to explicitly leverage the rich temporal and contextual information available in the video. To address this limitation, we propose a Video-Guided Post-ASR Correction (VPC) framework that uses a Video-Large Multimodal Model (VLMM) to capture video context and refine ASR outputs. Evaluations on a TV-series benchmark show that our method consistently improves transcription accuracy in complex multimedia environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 24 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

18 of 24 references.

References · 18

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  6. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  7. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  8. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  9. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  10. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 287
  11. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  12. 2020VIOLIN: A Large-Scale Dataset for Video-and-Language Inferencecited by 41
6 more
  1. 2023AVFormer: Injecting Vision into Frozen Speech Models for Zero-Shot AV-ASRcited by 16
  2. 2022AVATAR: Unconstrained Audiovisual Speech Recognitioncited by 13
  3. 2024Look, Listen and Recognise: Character-Aware Audio-Visual Subtitlingcited by 8
  4. 2023Generative error correction for code-switching speech recognition using large language modelscited by 7
  5. 2025Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correctioncited by 7
  6. 2023VILAS: Exploring the Effects of Vision and Language Context in Automatic Speech Recognitioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LiveCC: Learning Video LLM with Streaming Speech Transcription at Scale
  2. 2024Exploring the Potential of Multimodal LLM with Knowledge-Intensive Multimodal ASR
  3. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
  4. 2024Enhancing Audiovisual Speech Recognition through Bifocal Preference Optimization
  5. 2025Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models
  6. 2024Character-aware audio-visual subtitling in context
  7. 2026SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition
  8. 20262nd of the 5th PVUW MeViS-Audio Track: ASR-SaSaSa2VA