# Audio-visual speech recognition

Area: [Speech and audio language models](https://mlsanity.com/speech/index.md). HTML: https://mlsanity.com/speech/audio-visual-speech-recognition/

## Key facts

- Audio-visual speech recognition: 283 papers on arXiv since 2021 (data through 19 Oct 2026).
- 118 of its 283 mapped papers (42%) cite “Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction” (2022, arXiv 2201.02184).
- 115 of its 283 mapped papers (41%) cite “End-to-end Audio-visual Speech Recognition with Conformers” (2021, arXiv 2102.06657).
- 3 new papers 2 Oct–19 Oct.

## Most cited here

- [Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction](https://mlsanity.com/paper/2201.02184/) (2022), cited by 118 papers here
- [End-to-end Audio-visual Speech Recognition with Conformers](https://mlsanity.com/paper/2102.06657/) (2021), cited by 115 papers here
- [Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels](https://mlsanity.com/paper/2303.14307/) (2023), cited by 84 papers here
- [Visual Speech Recognition for Multiple Languages in the Wild](https://mlsanity.com/paper/2202.13084/) (2022), cited by 55 papers here
- [Robust Self-Supervised Audio-Visual Speech Recognition](https://mlsanity.com/paper/2201.01763/) (2022), cited by 48 papers here
- [Sub-word Level Lip Reading With Visual Attention](https://mlsanity.com/paper/2110.07603/) (2021), cited by 42 papers here
- [Jointly Learning Visual and Auditory Speech Representations from Raw Data](https://mlsanity.com/paper/2212.06246/) (2022), cited by 42 papers here
- [Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation](https://mlsanity.com/paper/2406.10082/) (2024), cited by 35 papers here
- [Large Language Models are Strong Audio-Visual Speech Recognition Learners](https://mlsanity.com/paper/2409.12319/) (2024), cited by 33 papers here
- [Watch or Listen: Robust Audio-Visual Speech Recognition with Visual Corruption Modeling and Reliability Scoring](https://mlsanity.com/paper/2303.08536/) (2023), cited by 28 papers here

## New (2 Oct–19 Oct)

- [VoxDub: A Bilingual Lip-Synchronized Audio-Visual Dataset for Automatic Video Dubbing](https://mlsanity.com/paper/2610.25259/) (2026-10-18)
- [From Sound to Sense: An Anatomy of Semantic Supervision for Speech-Driven Visual Grounding in Remote Sensing Imagery](https://mlsanity.com/paper/2610.16364/) (2026-10-12)
- [ScreenASR-Bench: A Tiered Benchmark for Speech Recognition with Visual Context](https://mlsanity.com/paper/2610.16260/) (2026-10-12)