# VSRo-200: A Romanian Visual Speech Recognition Dataset for Studying Supervision and Multimodal Robustness

arXiv: https://arxiv.org/abs/2607.08112 · HTML: https://mlsanity.com/paper/2607.08112/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 9 Jul 2026.

## Builds on (most cited first, 21 of 21)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [LRS3-TED: a large-scale dataset for visual speech recognition](https://mlsanity.com/paper/1809.00496/) (2018)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [pyannote.audio: neural building blocks for speaker diarization](https://mlsanity.com/paper/1911.01255/) (2019)
- [Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation](https://mlsanity.com/paper/1804.03619/) (2018)
- [End-to-end Audio-visual Speech Recognition with Conformers](https://mlsanity.com/paper/2102.06657/) (2021)
- [S3FD: Single Shot Scale-invariant Face Detector](https://mlsanity.com/paper/1708.05237/) (2017)
- [Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels](https://mlsanity.com/paper/2303.14307/) (2023)
- [Lipreading using Temporal Convolutional Networks](https://mlsanity.com/paper/2001.08702/) (2020)
- [The Multilingual TEDx Corpus for Speech Recognition and Translation](https://mlsanity.com/paper/2102.01757/) (2021)
- [Visual Speech Recognition for Multiple Languages in the Wild](https://mlsanity.com/paper/2202.13084/) (2022)
- [Sample and Computation Redistribution for Efficient Face Detection](https://mlsanity.com/paper/2105.04714/) (2021)
- [LipNet: End-to-End Sentence-level Lipreading](https://mlsanity.com/paper/1611.01599/) (2016)
- [LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild](https://mlsanity.com/paper/1810.06990/) (2018)
- [Training Strategies for Improved Lip-reading](https://mlsanity.com/paper/2209.01383/) (2022)
- [Visual Speech Recognition for Languages with Limited Labeled Data using Automatic Labels from Whisper](https://mlsanity.com/paper/2309.08535/) (2023)
- [A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learning](https://mlsanity.com/paper/2202.13403/) (2022)
- [End-to-End Lip Reading in Romanian with Cross-Lingual Domain Adaptation and Lateral Inhibition](https://mlsanity.com/paper/2310.04858/) (2023)
- [RO-N3WS: Enhancing Generalization in Low-Resource ASR with Diverse Romanian Speech Benchmarks](https://mlsanity.com/paper/2603.02368/) (2026)

## Abstract (arXiv)

We introduce VSRo-200, the first large-scale dataset for visual speech recognition (lip reading) in Romanian, comprising 200 hours of real-world podcast videos. All samples are annotated with pseudo-labels generated by a fine-tuned Romanian ASR model, while a subset of 100 hours is additionally transcribed by humans, enabling controlled analysis of supervision quality under a unified framework. Building on this dataset, we establish a benchmark for visual speech recognition in low-resource settings. We systematically study the impact of supervision quality, showing that while human annotations provide better performance at fixed data scales, pseudo-labels enable continued improvements through scalability. We further evaluate robustness under domain shift using curated out-of-distribution (OOD) test sets, and analyze audio-visual speech recognition (AVSR) under noisy conditions, where multimodal fusion significantly improves robustness compared to audio-only models. Finally, we demonstrate that representations learned on VSRo-200 transfer effectively to the LRRo benchmark for isolated word recognition, substantially outperforming previously reported results. Overall, VSRo-200 provides a new testbed for studying supervision, domain generalization, and multimodal fusion in low-resource visual speech recognition.