# ScreenASR-Bench: A Tiered Benchmark for Speech Recognition with Visual Context

arXiv: https://arxiv.org/abs/2610.16260 · HTML: https://mlsanity.com/paper/2610.16260/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 34 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 29 of 29)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [Qwen3.5-Omni Technical Report](https://mlsanity.com/paper/2604.15804/) (2026)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale](https://mlsanity.com/paper/1808.10583/) (2018)
- [FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs](https://mlsanity.com/paper/2407.04051/) (2024)
- [Qwen3-ASR Technical Report](https://mlsanity.com/paper/2601.21337/) (2026)
- [MiMo-Audio: Audio Language Models are Few-Shot Learners](https://mlsanity.com/paper/2512.23808/) (2025)
- [Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexity](https://mlsanity.com/paper/2607.00248/) (2026)
- [SlideSpeech: A Large-Scale Slide-Enriched Audio-Visual Corpus](https://mlsanity.com/paper/2309.05396/) (2023)
- [FireRedASR2S: A State-of-the-Art Industrial-Grade All-in-One Automatic Speech Recognition System](https://mlsanity.com/paper/2603.10420/) (2026)
- [AVATAR: Unconstrained Audiovisual Speech Recognition](https://mlsanity.com/paper/2206.07684/) (2022)
- [Listen, Look and Deliberate: Visual context-aware speech recognition using pre-trained text-video representations](https://mlsanity.com/paper/2011.04084/) (2020)
- [MOSS-Audio Technical Report](https://mlsanity.com/paper/2606.01802/) (2026)
- [SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processing](https://mlsanity.com/paper/2601.09385/) (2026)
- [SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognition](https://mlsanity.com/paper/2401.09759/) (2024)
- [Chinese-LiPS: A Chinese audio-visual speech recognition dataset with Lip-reading and Presentation Slides](https://mlsanity.com/paper/2504.15066/) (2025)
- [Do Slides Help? Multi-modal Context for Automatic Transcription of Conference Talks](https://mlsanity.com/paper/2510.13979/) (2025)
- [Index-ASR Technical Report](https://mlsanity.com/paper/2601.00890/) (2025)
- [StepAudio 2.5 Technical Report](https://mlsanity.com/paper/2605.23463/) (2026)
- [Seeing the Context: Rich Visual Context-Aware Speech Recognition via Multimodal Reasoning](https://mlsanity.com/paper/2603.07263/) (2026)
- [Qwen-Audio-3.0-ASR Technical Report](https://mlsanity.com/paper/2609.07549/) (2026)
- [VAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models](https://mlsanity.com/paper/2510.08618/) (2025)
- [OCR-Enhanced Multimodal ASR Can Read While Listening](https://mlsanity.com/paper/2601.18393/) (2026)

## Abstract (arXiv)

Current ASR models perform well on general benchmarks but still struggle with acoustically ambiguous words. Visual context can aid recognition in screen-based scenarios, yet existing benchmarks provide limited insight into the benefits of different types of visual evidence. To address this gap, we introduce ScreenASR-Bench , a tiered benchmark with three progressively challenging levels: Explicit Textual Evidence (ETE), based on direct on-screen text; Explicit Visual Evidence (EVE), using visible content rather than on-screen text; and Implicit Semantic Evidence (ISE), requiring scene-level inference. Extensive experiments on ASR models, Large Audio-Language Models (LALMs) and Omni models show that incorporating visual context substantially improves recall over the audio-only setting across all three model families. The gains are largest for ETE and smallest for ISE, indicating that models leverage direct visual evidence more effectively than indirect semantic cues. To facilitate future research on visually grounded speech recognition, we publicly release ScreenASR-Bench.